# Overview
Source: https://www.omnigraph.dev/docs
Omnigraph is a lakehouse native graph engine built for context assembly. It
acts as the operational state and coordination layer for agents, with
git-style versioning and branching, multimodal retrieval, and object storage
native deployment on the [Lance](https://github.com/lance-format/lance) format.
## Quick install
```bash
curl -fsSL https://raw.githubusercontent.com/ModernRelay/omnigraph/main/scripts/install.sh | bash
```
Or with Homebrew:
```bash
brew tap ModernRelay/tap
brew install ModernRelay/tap/omnigraph
```
New here? Follow the [install guide](https://www.omnigraph.dev/docs/getting-started/install.md), or clone a
[starter cookbook](https://github.com/ModernRelay/omnigraph-cookbooks) to get a
working graph with seed data in one command.
## Explore the docs
# Changes, Diffs and Change Feeds
Source: https://www.omnigraph.dev/docs/branching/changes
OmniGraph can describe what one commit changed, compare any two points in
history, review what a branch changed, or deliver an ordered feed of changes
on a branch. Every surface reports logical nodes and edges, not storage
details, in one change shape.
HTTP calls follow the [HTTP contract](https://www.omnigraph.dev/docs/operations/server.md#http-contract),
including the required request header and response check before consuming a
JSON or streamed body.
## Inspect one commit
```bash
omnigraph commit changes --target ./graph.omni --json
```
The equivalent HTTP route is
`GET /graphs/{graph_id}/commits/{commit_id}/changes`. The commit is compared
with its first parent. Each response includes:
* `cause`: the commit id, parent, optional merged parent, authored branch,
optional actor, and authorship time in Unix microseconds.
* `changes`: inserts with `after`, updates with `before` and `after`,
and deletes with `before`. Each image contains `properties`; an edge image
also contains `endpoints: {from, to}`. A Blob property holds its export
value (`base64:…` or an external URI), or, for a stored reference to a byte
range of an object, `{"uri", "offset", "length"}` (see
[Blobs](https://www.omnigraph.dev/docs/schema/blobs.md)).
Filter with repeatable `--kind node|edge`, `--type `, and
`--op insert|update|delete` options. A page holds at most 8,192 changes; the
HTTP routes default to 1,000. Results are deterministic by entity kind, type
identity, logical id, and operation.
Large results are paginated. The CLI normally follows every
`next_page_token` and asks for 8,192 changes per page, because every page
restarts its comparison and fewer, larger pages cost less; passing
`--page-token` fetches exactly one page, of 1,000 changes unless `--limit`
sets the size. A page token continues that one commit result and is not a
change-feed cursor.
A parentless commit returns `409` with reason `parentless_commit`. A diff
that crosses a schema boundary returns `409` with reason
`schema_boundary`; take a new baseline instead of treating either response
as an empty change.
## Compare two points
```bash
omnigraph diff --from --to main --target ./graph.omni --json
```
Each side is a branch name, meaning its head when the diff starts, or a commit
id: a published commit, an unpublished proposal, or an unattached snapshot. A
string that names both a branch and a commit is refused; name the branch's
head by its commit id instead. The equivalent HTTP route is
`GET /graphs/{graph_id}/diff?from=&to=`, and the GQ statement
`branch diff to ` answers the same comparison at `POST /query`,
a commit id quoted (`branch diff "" to main`).
The result is the net change from the `from` state to the `to` state, in the
`commit changes` change shape and order, with `from_commit_id` and
`to_commit_id` naming the exact commits the sides resolved to. There is no
`cause`: a range of commits has no single author. Intermediate history is not
replayed, so an update that was later reverted produces nothing, and a delete
followed by an equal reinsert produces nothing. A diff from a commit's first
parent to the commit equals that commit's `commit changes`, without the cause.
Comparing two branch heads also shows what the target changed since the
branches diverged, reversed. To review a branch, use the branch form below.
The kind, type and operation filters, `--limit` and `--page-token` work as for
`commit changes`. A page token keeps the commits the first page resolved, so
later pages answer for them even after the branches move. Each continuation is
checked against the request's sides, on the server and in the embedded CLI
alike. A side spelled as a commit id reaches only that commit. A branch side
reaches its head now, any of its earlier heads within its last 1,024 commits,
and any commit the caller may read as a side of its own. A token naming any
other commit is refused with `400`; start the diff again without it. A diff
across a
schema change returns `409` with reason `schema_boundary`, and a side whose
history was reclaimed returns `410` with `change_feed_gap`.
Types unchanged between the two points are skipped. A changed type is
compared row by row. For a single commit the engine reads the fragments that
commit wrote. Across several commits it pairs the two versions' live rows by
Lance's stable row id and reads only the rows whose version stamp differs or
that exist on one side alone, so the cost follows the changed rows and the
fragments that hold them, not the table's size, and stays flat across a
compaction. That pairing applies when the `from` side is on the `to` side's
history with no merge commit after it: a branch review whose merge base is on
the source's first-parent history, and a two-point diff whose `to` names a
branch whose first-parent chain holds `from`. A merge, a fast forward included,
can give a type the merged branch's rows, whose history never passed through
`from`. A diff of two diverged heads compares every row of each changed type,
and so do a review of a branch that has merged its target since the fork, a
window that crosses a merge commit, and a merge commit's own changes, except
for a type the merge wrote in one step. The `change_lineage`
[session setting](https://www.omnigraph.dev/docs/queries-search/settings.md) selects the comparison for every
change read: `on` (the default) pairs rows by id, `off` compares every row,
and `verify` runs both and fails the page where they disagree, which a debug
build does by default.
## Review a branch
```bash
omnigraph branch diff feature --into main --target ./graph.omni --json
omnigraph branch merge-base feature --into main --target ./graph.omni --json
```
`branch diff` reports what `feature` changed since its merge base with `main`
(`main` when `--into` is omitted): the net change from the merge base to the
head of `feature`. Changes `main` made after the merge base do not appear. The
response also names the base as `base_commit_id`. `branch merge-base` prints
that commit alone, the one a merge of `feature` into `main` would classify
against. While `feature` has never merged `main` in, the base is on its
first-parent history, and `changes poll --branch feature --start
after:` replays its commits since then; a merge commit in that
range still carries the changes it brought in. A base off `feature`'s
first-parent history is refused as a start: for example, after `main` is
merged into a `feature` that had diverged from it, the base is the `main`
commit that merge brought in. `branch diff` gives the net change from any
base.
The HTTP routes are `GET /graphs/{graph_id}/branches/diff?source=&target=` and
`GET /graphs/{graph_id}/branches/merge-base?source=&target=`; the GQ statements
are `branch diff feature into main` and `branch merge-base feature into main`.
A statement answers rows through `POST /query` or `omnigraph query -e`. A
`branch diff` row has `kind`, `type` (the type's name), `id`, `op`, `before`
and `after`; the compared commits and the token for the next page travel in
the answer's `page` object, and `after page ""` (then `limit `)
continues it. A `branch merge-base` row names `source`, `target`, both heads
and `merge_base_commit_id`. Statements take no kind, type or operation filter
yet; use the routes or CLI commands for those. To see whether
the merge would succeed, run it as a [dry run](https://www.omnigraph.dev/docs/branching/merge.md#preview-a-merge).
## Follow a branch
```bash
# Start at the current head. Existing history is not replayed.
omnigraph changes poll --start now --target ./graph.omni --json
# Replay the branch's first-parent history.
omnigraph changes poll --start beginning --target ./graph.omni --json
# Continue after a known commit, or resume from a saved cursor.
omnigraph changes poll --start after: --target ./graph.omni --json
omnigraph changes poll --cursor --target ./graph.omni --json
```
`now` is the default when neither `--start` nor `--cursor` is supplied.
Use `--branch` to follow a branch other than `main`. The same kind, type,
operation filters, and `--limit` supported by `commit changes` are available
here.
Each poll captures a fixed branch head and returns complete commits in
first-parent order. A commit with no logical entity changes may still appear
as an empty block and advance the feed.
The durable `cursor` appears only on the terminal page, after every returned
commit block is complete. A page ending partway through a commit has only a
`next_page_token`. On the terminal page, `caught_up` says whether the poll
reached its captured head. The CLI consumes feed page tokens itself and prints
the cursor only after reaching that terminal page.
Delivery is at least once: retrying a cursor may replay the next complete
commit. Apply each block idempotently by `graph_commit_id`, then persist the
terminal cursor atomically with the applied blocks.
Cursors are opaque and caller-owned; the server stores no consumer position.
A cursor is bound to its graph, branch lifetime, and filter scope. Reusing it
with a different scope, or using a page token as a cursor, returns `400`.
The HTTP route is `GET /graphs/{graph_id}/changes`. Its `cursor`, `start`,
and `page_token` parameters are mutually exclusive.
## Recover from a retention gap
A cursor does not prevent `cleanup` from reclaiming old history. When a feed,
commit diff or diff can no longer be read, the server returns `410` with
`change_feed_gap`, including `first_unreadable_commit_id` and, when
available, the rejected `cursor`. Retrying the same cursor cannot close the
gap; install a new baseline:
```bash
omnigraph changes baseline --out snapshot.jsonl --target ./graph.omni --json
```
If a poll has already accumulated complete readable blocks before it reaches a
gap, it first returns those blocks with `caught_up: false`; polling the
returned cursor then produces the deterministic `410`.
The HTTP equivalent is `POST /graphs/{graph_id}/changes/baseline`. Send `{}`
for the default `main` scope or include branch and filter fields. The
`application/x-ndjson` response streams an exact entity snapshot followed by
one terminal record:
```json
{"baseline":{"snapshot_commit_id":"...","resume_cursor":"..."}}
```
Snapshot records use the export line format, and a Blob property holds the
same value shapes as a change image. That includes the
`{"uri", "offset", "length"}` object for a stored reference to a byte range of
an object, which export refuses; a snapshot holding one does not reload with
`load` (see [Blobs](https://www.omnigraph.dev/docs/schema/blobs.md)).
An interrupted stream has no terminal record and therefore no usable cursor.
Install the complete snapshot durably before saving `resume_cursor`; the
resumed feed begins with commits after the captured snapshot.
Baseline kind and type filters select the snapshot contents. An operation
filter applies only to the resumed feed. Baselines require the `export`
policy action; commit changes, diffs and feed polling require `read`. A diff
needs `read` on the branch each side resolves to; a side the actor cannot read
answers `404` exactly as a side that names nothing, whatever the other side
names. The engine reads a commit whose branch was deleted while a live
branch retains it (a stacked branch's merge base after its parent was merged
and deleted): diffs, commit changes, reads at the commit and the feed open the
retired incarnation by its exact commit. The HTTP `commit changes` route still
needs `read` on the branch that wrote the commit, so a policy that does not
grant the deleted name answers `404` there. A commit that no live branch retains,
or whose recorded incarnation was reclaimed, answers the same `404` as a
commit that names nothing; the error names the spelling the caller used, never
the deleted branch. Branch creation from such a commit still refuses with
`branch_deleted`.
`branch diff` and `branch merge-base` need `read` on both branches, every
page included, and answer `403` otherwise. On a later page, a commit the
token names that its side no longer resolves to needs `read` on the branch
that wrote it, unless it was an earlier head of the side's branch, such as a
fork's first head that `main` wrote. An unknown or unreadable one answers the
same `400`.
The CLI's durable `--out` installation is available on POSIX platforms. The
file contains only snapshot entity records, not the terminal handshake. The CLI
syncs and atomically replaces it before printing
`{"snapshot_commit_id":"...","resume_cursor":"..."}` to JSON stdout. Concurrent
captures to the same path stay serialized until the cursor output is flushed.
A cursor-output failure returns an error; the installed snapshot remains in place.
# Branches, Commits, and History
Source: https://www.omnigraph.dev/docs/branching
A branch is an isolated, durable graph history. Use branches to prepare and
review a multi-step change without exposing intermediate results on `main`.
## Branch workflow
```bash
# Create an isolated branch from main.
omnigraph branch create review/add-benchmark --from main --target ./graph.omni
# Write and inspect it.
omnigraph load --data benchmark.jsonl --mode append \
--branch review/add-benchmark --target ./graph.omni
omnigraph query sources_for_claim --query queries.gq \
--params '{"claim":"lower-latency"}' \
--branch review/add-benchmark --target ./graph.omni
# Publish its result to main, then remove the source branch.
omnigraph branch merge review/add-benchmark --into main --delete-branch \
--target ./graph.omni
```
List and delete branches with:
```bash
omnigraph branch list --target ./graph.omni
omnigraph branch delete review/abandoned --target ./graph.omni
```
`branch list --detailed` (served: `GET /branches?detailed=true`; MCP:
`branches_list` with `detailed`) adds a `details` array. Per branch it reports
`head`, the commit a read sees; `from` and `from_commit_id`, the branch and
commit it started from, as its create receipt named them (a later merge moves
the merge base, not these); `created_at`, recorded to the second; and
`updated_at`, the newest commit's time, or `created_at` while the branch has
written none. Times are Unix microseconds. `main` omits `from`,
`from_commit_id` and `created_at`. A branch deleted and created again reports
its new creation. Each branch costs a read of its own manifest, two once it has
written a commit, whatever its history, and the branches are not read as one
cut. The served listing needs the `read` that `GET /branches` needs, and has
details only for branches the caller may `read`.
Creating a branch defaults to `main` when `--from` is omitted. To start a
branch at a past commit instead of a branch head, use `--from-commit`; see
[start a branch at a commit](#start-a-branch-at-a-commit). A load can create a
missing target branch by combining `--branch ` with `--from `.
Each branch operation is also a GQ statement, so a client that already sends
`.gq` source needs no second transport. The three control writes go to
`POST /mutate`, and the listing and the branch reads to `POST /query`, each
with no request target, name, or parameters:
```text
branch create "review/add-benchmark" from main
branch create "fix/import" from commit ""
branch merge "review/add-benchmark" into main
branch delete "review/abandoned"
branch list
branch diff "review/add-benchmark" into main
branch merge-base "review/add-benchmark" into main
branch merge "review/add-benchmark" into main dry run
```
`from` defaults to `main` and `into` defaults to `main`. A name outside the
identifier alphabet (a lowercase letter or `_`, then letters, digits, or `_`),
such as one containing `/`, `-`, or `.`, is quoted, as above; `main`, `b0`,
and `_x` are bare. A control write answers a
`ChangeOutput` whose `outcome.kind` is `created`, `deleted`, or `merged`
(`outcome.merge` holds the merge result), with zero affected counts; a merge
that publishes a commit reports it in `commit`. `branch list` answers one row
per branch in column `name`, sorted by name. Each statement runs under the same
policy check as its HTTP route (`POST /branches`, `DELETE /branches/{branch_name}`,
`POST /branches/merge`, `GET /branches`), and `branch list` is a read, so it is
refused on `POST /mutate` as a control write is refused on `POST /query`. The
branch reads `branch diff`, `branch merge-base` and `branch merge … dry run`
answer rows as `branch list` does and take their routes' checks; see
[reviewing a branch](https://www.omnigraph.dev/docs/branching/changes.md#review-a-branch) and
[previewing a merge](https://www.omnigraph.dev/docs/branching/merge.md#preview-a-merge). A bare `dry run` after the
names is the clause, so a branch named `dry` is quoted.
Branches are cheap until written: unchanged data remains shared with the source.
A branch remains after a normal merge, so prefer `--delete-branch` or delete it
when review is complete. Live branches retain the history they depend on and can
prevent `omnigraph cleanup` from reclaiming old data.
## Branch names
Branch names may contain `/`, but live names must not be path prefixes of one
another. For example, `review` and `review/alice` cannot coexist. `main` is
reserved. A parent branch can be deleted while descendants remain. A path segment
may not end in `.` followed by 26 upper-case letters and digits: that shape is
reserved for OmniGraph's internal per-life branch identity. Ordinary dotted
names such as `release.1.2` are fine.
A branch name is used exactly as written. A name that is empty, has leading or
trailing whitespace, or contains a control character is refused wherever a
branch is named: the branch a read or write targets, `from`, merge sources and
targets, branch create and delete, Blob writes, change feeds and branch
statements. The server answers `400 Bad Request` before authorization, the
embedded engine returns a bad-request error before any effect, and the CLI
reports either error. Names are never trimmed, so `main ` does not name
`main`.
Deleting a branch and creating another with the same name yields a fresh
branch lifetime. Readers that captured the deleted branch fail with a typed
error rather than seeing the replacement. Deletion retires the logical name
and leaves native manifest history and table storage for explicit `omnigraph cleanup`; it starts no background table-reclaim work. Cleanup preserves data still needed by
other branches and their underlying histories.
Branch-control operations are safe across handles in one writer process. Do not
run branch create/delete control concurrently from separate writer processes
against the same graph.
## Atomicity model
OmniGraph does not provide connection-scoped `BEGIN` and `ROLLBACK`.
| Scope | Guarantee |
| ---------------------------- | --------------------------------------------------------------------------------------------------------------------------- |
| One mutation query | All statements publish as one commit or none become visible. |
| One load request | The complete batch publishes as one commit or none becomes visible. |
| Several commands on a branch | Each command that publishes a change is a durable branch commit. Earlier commands are not rolled back if a later one fails. |
| Branch merge | The resulting source state becomes visible on the target in one atomic commit. |
Deleting an abandoned branch discards that workspace from normal access, but it
is an explicit lifecycle action rather than transaction rollback.
## Commits
Every write that publishes a change records a graph commit. A successful
mutation that matches no entities publishes no commit. A commit includes its
id, parent, branch, actor when known, and timestamp. Merge commits have two
parents.
```bash
omnigraph commit list --target ./graph.omni --branch main
omnigraph commit show --target ./graph.omni
```
`commit list` is newest first. Omitting `--branch` shows history reachable from
`main`; selecting a branch includes the history inherited at its fork plus its
own commits.
A long history can be read a page at a time:
```bash
omnigraph commit list --target ./graph.omni --branch main --limit 50 --json
omnigraph commit list --target ./graph.omni --branch main --page-token --json
```
`--limit` (1 to 1,000) returns one page with `head_commit_id`, the head the
listing resolved, and `next_page_token` while older commits remain. Pass the
token back with the same `--branch` for the next page, of 100 commits unless
`--limit` sets the size. Later pages answer for that head even after the
branch moves, so a page never repeats or skips a commit. A page reads only the
history it returns; without `--limit` or `--page-token` the whole history is
read and listed at once. Served under a policy, a caller whose `read` does
not cover every branch continues only from a commit of the branch's history,
and is refused with `400` otherwise: a commit the branch still holds, or one a
branch the caller may read published, costs nothing more, and any other is
looked for down the branch's history, which costs what the whole listing
costs. Served, the same listing is
`GET /commits?listing=history&branch=&limit=` continued with
`page_token=`; omitting `listing` also selects history. Paging applies
to a branch's history, not to `--unpublished` or `--unattached`.
HTTP `listing` accepts `history`, `unpublished` or `unattached`; non-history
listings refuse `limit` and `page_token`. The retired query keys `unpublished`
and `unattached` return `400 bad_request` even when false or combined with
`listing`; omit them or use `listing=history` for ordinary history.
### Start a branch at a commit
A branch can start at a published graph commit instead of a branch's head. It
holds that commit's rows and history, as if it had been created while the
commit was the head. Use it to try a fix or a revert where a change was made,
then merge the branch as usual; the commit is its merge base.
```bash
omnigraph commit list --target ./graph.omni --branch main --json
omnigraph branch create fix/import --from-commit --target ./graph.omni
```
The statement is `branch create "fix/import" from commit ""`, and
`POST /branches` takes the id in `from_commit`, which excludes `from` (a
request with both answers `400`). An id no commit has answers `404`. The
receipt names the branch the new
branch was forked off in `from`, for a commit start the branch that wrote the
commit, and the commit it starts at in `from_commit_id`. Every receipt carries
`from_commit_id`: for a start from a branch it is that branch's head when the
branch was created. A branch starts only at a commit whose whole snapshot is
still there under the graph's current schema; otherwise nothing is created,
and the error's `code` is `branch_start_refusal`, with `action: fix`:
| `from_commit` names | Answer | `branch_start_refusal.reason` |
| --------------------------------------------------------------------------------------------- | ------ | -------------------------------------------------------------------- |
| A commit whose table versions `cleanup` reclaimed, wholly or in a run that stopped part way | `410` | `not_retained`, with the type in `type_name` |
| A commit written under an earlier schema than the graph's | `409` | `schema_changed` |
| A commit written before the storage upgrade | `409` | `earlier_storage_format` |
| A commit of a branch that was deleted, even if a branch has since been created under its name | `409` | `branch_deleted`, with the branch in `branch` |
| An [unpublished commit](#unpublished-commits) | `409` | `unpublished_commit`; publish it, then start at the published commit |
| An [unattached snapshot](#unattached-branches) | `409` | `unattached_snapshot`; it never becomes branch history |
While the new branch lives, `cleanup` keeps the commit's table versions, so a
later start at the same commit succeeds too.
### Unpublished commits
A mutation or a merge written `as commit` records its result as an unpublished
commit: the branch head is unchanged, the result is readable at the commit's
id (`--snapshot `), and the commit lists with `published: false`.
```graphql
query propose_bob() as commit {
insert Person { name: "bob", age: 41 }
}
branch merge feature into main as commit
commit publish ""
commit drop ""
```
`commit publish` installs the commit on the branch it recorded as its target.
The published commit keeps the actor and timestamp of the write that created
it; the publish records no publisher. A published commit is answered as it
stands by a repeat publish and is never dropped. An unpublished commit expires
once an ordinary write moves its target branch's head past the commit's
parent, or when that branch is deleted or recreated, and can then only be
dropped: a publish is refused (a moved head fails its precondition), reads at
its id (`--snapshot`, `commit show`, `commit changes`) answer `not found`, and
it leaves the `--unpublished` listing.
```bash
omnigraph branch merge feature --into main --as-commit --target ./graph.omni
omnigraph commit list --target ./graph.omni --branch main --unpublished
omnigraph commit publish --target ./graph.omni
omnigraph commit drop --target ./graph.omni
```
Served, the same control writes are `POST /commits/{commit_id}/publish`
(the published commit), `DELETE /commits/{commit_id}` (204), and
`GET /commits?listing=unpublished&branch=`; `POST /branches/merge` takes
`as_commit`, which excludes `delete_branch`. Publishing and dropping run under
the `commit_publish` and `commit_drop` policy actions on the recorded target
branch ([Policy](https://www.omnigraph.dev/docs/operations/policy.md)).
### Unattached branches
A mutation written `as unattached branch` records its result as an unattached
snapshot: the branch head is unchanged, the result is readable at the
snapshot's id (`--snapshot `), and the snapshot never publishes. A second
write `on ""` (or `on $base` with a `String` parameter) builds on that
snapshot and returns a new id; the caller keeps the latest id as its tip. The
snapshot's branch is the one the first write ran on (`main` when the request
names none); a write `on` a snapshot runs on that recorded branch, and a
request that names another branch is refused. Two writes on one base each get
their own snapshot and see nothing of each other's.
```graphql
query add_person($name: String) as unattached branch {
insert Person { name: $name }
}
query add_address($id: String, $base: String) as unattached branch on $base {
insert Address { owner: $id, city: "Delft" }
}
```
A write that changes nothing returns `commit: null` and creates no snapshot;
the caller keeps its previous tip. The receipt's `commit` carries
`kind: "unattached"`, `published: false` and `root_commit_id`, the first
snapshot of the chain (its own id on that first snapshot); `parent_commit_id`
is the base snapshot, or the branch head the first write ran on. A base must
be a live unattached snapshot: a published commit or an unpublished commit is
refused.
```bash
omnigraph commit list --target ./graph.omni --branch main --unattached
omnigraph query people --query queries.gq --snapshot --target ./graph.omni
omnigraph commit drop --target ./graph.omni
```
`commit list --unattached` shows the branch's live snapshots, newest first,
flat; `parent_commit_id` tells the chains apart. `commit drop` of a snapshot
also drops every snapshot built on it; dropping the first one discards the
whole chain, and a snapshot built on another chain is untouched. `commit
publish` of an unattached snapshot is refused with HTTP 409 and the error
code `unattached_cannot_publish`; nothing is written. Every snapshot of a
chain expires when an ordinary write moves the branch head past the head the
first write captured, or when the branch is deleted or recreated: expired
snapshots answer `not found` to reads and writes and leave the listing. Dropping or expiring
a snapshot promises no storage reclamation; `cleanup` reports what it retains.
Served, the listing is `GET /commits?listing=unattached&branch=`
and the drop is the same `DELETE /commits/{commit_id}` (204). Writing needs
`change` on the recorded branch; listing and reading a snapshot need `read` on it
([Policy](https://www.omnigraph.dev/docs/operations/policy.md)).
## Historical reads
Every read targets either a live branch or an immutable graph commit. Take a
`graph_commit_id` from `commit list --json` and pass it as the snapshot:
```bash
omnigraph query sources_for_claim --query queries.gq \
--params '{"claim":"lower-latency"}' \
--snapshot --target ./graph.omni
```
A query stays on one snapshot for its entire lifetime. Wildcard traversal is
refused on an explicit historical target because historical schema membership
is not reconstructed. Explicit edge alternatives retain named-edge historical
read rules.
A historical query is written against the current schema. A type renamed since
the commit still reads under its current name, but a property reads only where
the commit stores that same property under its current name. A property added
since the commit, renamed since, or dropped and re-added with the same name is
refused at that commit, naming the property, rather than answering with null or
with the dropped property's values. A whole-node `return { $p }` reads every
current property, so it is refused at commits made before a property was added.
Table data written before v0.10 records no property identity, so a commit that
reads such data refuses its properties too, unless that data is still the
table's current version.
Historical reads can eventually fail after destructive cleanup removes the
versions that commit needs. Branch deletion can likewise end access to branch-only history.
## Changes and feeds
Inspect the logical changes made by one commit, compare two points, or review
what a branch changed since it left `main`:
```bash
omnigraph commit changes --target ./graph.omni --json
omnigraph diff --from --to main --target ./graph.omni --json
omnigraph branch diff feature --target ./graph.omni --json
```
For continuous consumption, `omnigraph changes poll` follows complete commits
on one branch and returns an opaque resume cursor. If cleanup has reclaimed
required history, create a coherent snapshot and new cursor with
`omnigraph changes baseline`.
See [Changes, Diffs and Change Feeds](https://www.omnigraph.dev/docs/branching/changes.md) for filters, pagination,
cursor checkpointing, and `410` retention-gap recovery.
See [Merging Branches](https://www.omnigraph.dev/docs/branching/merge.md) for merge outcomes and conflict handling.
# Merging Branches
Source: https://www.omnigraph.dev/docs/branching/merge
A merge integrates one branch into another and publishes the target change
atomically across the whole graph.
```bash
omnigraph branch merge review/2026-04-25 --into main \
--target ./graph.omni
```
The source is positional. `--into` defaults to `main`.
The same merge as a GQ statement, sent to `POST /mutate` with no request
target, name, or parameters:
```text
branch merge "review/2026-04-25" into main
```
`into` defaults to `main`; a name with `/`, `-`, or `.` is quoted, a bare
identifier such as `main` is not. The answer is a `ChangeOutput` with
`outcome.kind = "merged"` and `outcome.merge` one of `already_up_to_date`,
`fast_forward`, or `merged`; after a `fast_forward` or `merged` result,
`commit` identifies that merge's own publication, even if another writer has
since advanced the target. The statement has no `--delete-branch`
composition: follow it with `branch delete `. A merge statement takes
no commit precondition -- `POST /mutate/if-graph-commit` and `--if-commit`
refuse one beside it -- and no front offers a conditional merge today.
A merge preserves changes already integrated into a branch when you later
merge that branch back. For example, after merging a new edge from `main`
into `review`, editing only a node on `review` and merging it into `main`
preserves that edge. A merge whose source and target tables reached the same
version count on different branches fast-forwards; earlier releases refused
it with a `table version … already exists` error.
## Outcomes
* **Already up to date**: the target already contains the source changes.
* **Fast-forward**: the target still has the merge base's schema and table
versions, so it takes the source's state. The check reads state, not
history: a target merged again from a branch it fast-forwarded before, with
no write in between, fast-forwards again. One exception: when a
fast-forward into `main` carried a table whose changes cancel out on the
branch, or a table on a fork left by a branch created before storage format
v11, `main` ends on its own version of that table, so a later merge from
that branch can report **Merged**. The merged data is the same.
* **Merged**: both branches changed, so OmniGraph performs a three-way,
entity-level merge and creates a commit with two parents.
`branch merge --json`, the HTTP merge route and the GQ merge statement all
return `commit` with the exact commit id, manifest version, parents, branch,
actor and creation timestamp (Unix microseconds). Fast-forward returns the new
target commit; its merged parent identifies the source head. An already-up-to-date
merge publishes nothing and returns `commit: null`.
If a remote response is lost or incomplete, the merge may have committed. The
CLI does not resubmit it automatically. Inspect the intended target changes and
relevant history before deciding what to do next; a later head alone cannot
identify the missing merge receipt.
When only the source changed a table since the merge base, a merge into a
named branch can adopt that exact table snapshot without copying its rows.
This also works when the target already has its own table history. Tables
changed on both sides still use the normal three-way merge and constraint
checks.
The target can continue writing after adoption. Other live branches keep their
current table snapshots through those target writes and cleanup;
creating a replacement branch is unnecessary. Cleanup retains the underlying
histories they still need. Avoiding a table copy does not eliminate the reads
needed to validate a merge.
The source branch is unchanged by the merge. Use `--delete-branch` for the
normal review-branch lifecycle:
```bash
omnigraph branch merge review/2026-04-25 --into main --delete-branch \
--target ./graph.omni
```
Deletion happens after a successful merge and has its own authorization check.
If deletion is denied or the source still has descendants, the merge remains
successful and durable. The CLI exits 0 and warns in human output; HTTP and CLI
JSON preserve `commit`, report `branch_deleted: false` and include the structured
`branch_delete_error_details` (`error`, plus typed `code` and details where available).
Successful deletion reports `branch_deleted: true`; without `--delete-branch`,
both deletion fields are absent. v0.12 removes the old `branch_delete_error`
string. Inspect and retry deletion separately; do not replay a successful merge.
Deleting a branch is irreversible and may make branch-only commits unavailable.
It also releases old history for a later cleanup.
## Preview a merge
```bash
omnigraph branch merge feature --into main --dry-run --target ./graph.omni --json
```
A dry run runs the merge through classification and constraint validation,
then stops. It reports the outcome the merge would have now, or fails with the
same structured conflict list, and writes nothing: `commit` is `null` and
`dry_run` is `true`. It is a read of both branches, so it needs `read` on each
and no `branch_merge` grant. It refuses `--as-commit` and `--delete-branch`.
Over HTTP, send `dry_run: true` to `POST /graphs/{graph_id}/branches/merge`.
The GQ statement `branch merge feature into main dry run` is the same read at
`POST /query`: it answers one row with `source`, `target` and `outcome`, and a
conflict is the 409 a merge answers.
A dry run answers whether the merge would succeed. To see what it would change,
use [`branch diff`](https://www.omnigraph.dev/docs/branching/changes.md#review-a-branch). To review the exact result
before it lands, merge `--as-commit` and inspect the proposal with
`commit changes`; the proposal expires once the target branch moves.
## Conflicts
When both sides changed the same logical data incompatibly, the merge returns a
structured conflict list and publishes nothing.
| Kind | Meaning |
| ---------------------------- | ------------------------------------------------------------------- |
| `divergent_insert` | Both branches inserted the same id. |
| `divergent_update` | Both branches updated the same entity differently. |
| `delete_vs_update` | One branch deleted an entity the other updated. |
| `orphan_edge` | The result would contain an edge whose endpoint was deleted. |
| `unique_violation` | The result would violate uniqueness. |
| `cardinality_violation` | The result would violate edge cardinality. |
| `value_constraint_violation` | The result would violate an enum, range, or other value constraint. |
A merge classifies both sides against their merge base, the newest commit
both branches descend from, counting a merged branch as an ancestor: a
commit both branches descend from that no other such commit descends from.
When two such commits are each not an ancestor of the other, the base is the
one nearest the two heads (the fewest commits between it and the heads,
summed), then the one with the higher generation (its distance from the
first commit along the longest path), then the one with the smaller commit
id. Two branches that
each merged the same third branch therefore share that branch's commit as
their base, and an entity only one of them changed after that import merges
cleanly. The record of such a commit lives in the branch
it was merged from, and the merge can read it from a live branch or retained
retired history. Cleanup preserves the exact selected base and the retired
histories needed to resolve it. Recreating the logical branch name cannot
substitute the replacement branch's same-numbered snapshot.
Each conflict identifies the affected type and, when applicable, entity id. The
HTTP server returns conflicts with status `409`, the same answer for
`POST /branches/merge` and for a `branch merge` statement on `POST /mutate`.
Reconcile the data on one or both branches, then run the merge again.
### Edges inserted on both sides
Whether the same edge added on both branches converges depends on the edge
type's declared identity:
* No declaration: edge ids are generated, so each side's insert is its own
row and the merge keeps both. This is the documented multiset default;
parallel edges are legitimate data.
* `@unique(@src, @dst)`: each branch's write succeeds on its own, and the
merge reports `unique_violation`. Releases before edge keys provide this
guard as `@unique(src, dst)`; from 0.11 it is spelled
`@unique(@src, @dst)`.
* `@key(@src, @dst)`: both sides derive the same id, so identical inserts
converge to one row with no conflict. If the sides disagree on a non-key
property, the merge reports `divergent_insert` on the edge type with the
derived id as the entity id (for `@key(@src, @dst)` a JSON array such as
`["Alice","Bob"]`; the elements are the endpoint node ids, so they are
generated ids when the endpoint type declares no key). Reconcile it like any divergent insert: align the
property on one branch (insert the same key again with the agreed
values; the insert upserts), then merge again.
## Merge classification mode
The `merge_lineage` [session setting](https://www.omnigraph.dev/docs/queries-search/queries.md#session-settings)
(`request` scope; `set merge_lineage = off;` before the statement, `--set
merge_lineage=off`, or the request's `settings` field; process default
`OMNIGRAPH_MERGE_LINEAGE`) selects how a branch merge finds what changed. `on`
(the release default) discovers candidates from Lance version metadata —
fragment lists and deletion files — and compares candidate rows, for tables
both branches changed and for tables only the source changed (including a
fast forward), so its cost follows what changed rather than the table size. Known deleted
row positions use bounded direct reads. Candidate filtering may still scan data
when an applicable index is absent; a fail-closed
precondition gate falls back to the full three-way scan whenever any
assumption cannot be proven (Blob-bearing schema, differing schemas or storage
paths across the pins, version pins not matching the manifest entries, missing
stable identifiers or the exact-id primary-key contract, or a candidate set
past its byte budget). The table version a side pins need not descend from
the one the merge base pins, so merging a branch back after an earlier merge
still qualifies. `off` forces the full three-way scan everywhere — the
operational fallback if merge results are ever in question.
`verify` runs both, compares their decisions entity by entity, publishes the
scan's result, and fails the merge loudly on any divergence (the debug-build
default, used for validation; it costs both paths). A merge that succeeds
produces the same result in every mode; only cost differs. An unrecognized
value is refused where it arrives: in the environment it refuses startup, in
a request or a file it refuses that request.
Later writes create fresh table storage when needed and leave unused former
storage for explicit cleanup. They do not reclaim a previous table history as
part of preparing the write or merge.
## Validation limit
Before it publishes, a merge checks the target's constraints (value and enum
checks, uniqueness, edge endpoints and `@card`) against its delta: every row
the merge adds or changes since the merge base, and every id it removes. It
reads the delta from storage one batch at a time. A value that only a
single-row check reads, such as a long description or a property under
`@range`, is checked in its batch and then released, and vectors and Blob
payloads are not read at all. What the merge keeps until the checks finish
is the delta's keys: entity ids, the values of each `@key` and `@unique`
group, an edge's endpoints when they or its `@card` are checked, and the
removed ids.
Those keys are limited to 32 MiB per merge, counted as the bytes of their
values (a string's length, plus 4 bytes; a null counts nothing), whatever
buffers storage returned them in. The checks hold each key in lookup tables of
their own, so their memory is several times this count, not bounded by it.
The limit is fixed: `write_max_bytes` and `query_memory_bytes` do not change
it. The check sets no bound of its own on a row's width. A merge that writes
rows (for a table both branches changed) keeps its own `write_max_bytes`
allowance, as any write does, so a row wider than 32 MiB, written under a
raised value, needs that value again; a merge that adopts the source's table
by pointer writes no row.
It applies to the whole delta since the merge base, however the source's
commits were chunked, so a branch whose every load fit its own bounds can
still reach it. A merge past the limit publishes nothing and fails with
`resource limit exceeded for branch-merge retained validation key bytes`
(over HTTP, `413` with a `resource_limit` detail naming the resource, its
limit and the actual bytes; disposition `effects: none`, `action: fix`).
Merge such a branch in stages, each holding part of its commits:
1. List the source's commits and pick one partway along it:
`omnigraph commit list --branch --target ./graph.omni`.
2. Start a branch at that commit:
`omnigraph branch create --from-commit --target ./graph.omni`.
3. Merge ``, then merge the source. The first merge records the
commit as part of the target's history, so it becomes the base of the
second merge, whose delta holds only the later commits.
Use more stages if a part is still past the limit. A single commit whose own
keys pass the limit cannot be split this way; write that change again as
smaller commits. A fast-forward whose tables only gain nodes of types with no
`@unique`, `@range`, `@check` or enum property, and no edge type with a
`@card` minimum from them, needs no check and has no such limit.
## After a large merge
Indexes do not define merge correctness. Newly merged entities remain queryable even
when index coverage has not caught up, but some searches may scan them. Run
`omnigraph optimize` after a large merge to restore efficient layout and index
coverage.
A merge that writes rows is also bounded by the Blob payload it copies; see
[Blob limits](https://www.omnigraph.dev/docs/schema/blobs.md#limits).
See [Branches, Commits, and History](https://www.omnigraph.dev/docs/branching.md) for the complete branch workflow.
# CLI guide
Source: https://www.omnigraph.dev/docs/cli
Upgrading an existing installation? Start with the
[0.13 → 0.14 checklist](https://www.omnigraph.dev/docs/operations/upgrade-0.14.md) for before/after YAML
and command examples.
Select resources with `--target NAME|LOCAL_PATH`, or `defaults.target` when
omitted. Account operations use `--profile NAME` or `defaults.profile`.
These selectors are mutually exclusive. Source files select what to run; they
never choose where to run it.
```bash
# Open a graph directly; no registration needed.
omnigraph query sources_for_claim --query queries.gq \
--params '{"claim":"lower-latency"}' --target ./graph.omni
# Run a stored query through a registered server target.
omnigraph query sources_for_claim --target prod --graph knowledge \
--params '{"claim":"lower-latency"}'
# Remote endpoints require a registered target.
omnigraph graphs list --target prod
```
A bare word is a registered name. Write `./local` for a relative path named
`local`. A direct root or server needs `--graph ID` for graph operations unless
its registered target has `default_graph`. A direct graph already identifies
the graph and rejects `--graph`. Commands reject incompatible backend kinds
before making a request.
The CLI prints its resolved destination to stderr before mutations. An invalid
explicit target fails; it never falls back to the default. Folder context,
credentials and discovery responses cannot change the destination.
Run `omnigraph --help` for available flags. The
[CLI reference](https://www.omnigraph.dev/docs/cli/reference.md) covers the complete command surface, and
[operator configuration](https://www.omnigraph.dev/docs/cli/targets.md) is the primary reference for profiles,
targets, authentication providers and the breaking format change.
## Create, load, and query a graph
```bash
omnigraph init ./graph.omni --schema schema.pg
omnigraph load --target ./graph.omni --data evidence.jsonl --mode overwrite
omnigraph query sources_for_claim --target ./graph.omni \
--query queries.gq --params '{"claim":"lower-latency"}' --format table
omnigraph mutate add_source --target ./graph.omni --query mutations.gq \
--params '{"slug":"incident-review","title":"Incident review"}'
```
`init URI` is the positional shorthand for an inline direct graph target.
Other graph commands take their destination through `--target`.
`load` requires a mode:
* `overwrite` replaces each node or edge type represented in the batch. Types
absent from the batch are unchanged.
* `append` inserts new entities and rejects duplicate IDs.
* `merge` inserts new IDs and updates existing IDs.
For a stored query, omit `--query`; the positional name selects the query from
the selected backend's registry.
## Work with branches
```bash
omnigraph branch create review/new-data --from main --target ./graph.omni
omnigraph load --target ./graph.omni --data batch.jsonl --mode merge \
--branch review/new-data
omnigraph query inspect --target ./graph.omni --query review.gq \
--branch review/new-data
omnigraph branch merge review/new-data --into main --target ./graph.omni
```
Branch operations are also GQ statements. Statements name their branches
internally and take no `--branch`:
```bash
omnigraph mutate -e 'branch create "review/new-data" from main' --target ./graph.omni
omnigraph query -e 'branch list' --target ./graph.omni --format table
omnigraph mutate -e 'branch merge "review/new-data" into main' --target ./graph.omni
```
See [Branches and commits](https://www.omnigraph.dev/docs/branching.md).
## Session settings
`--set NAME=VALUE`, repeatable, gives a [session
setting](https://www.omnigraph.dev/docs/queries-search/queries.md#session-settings) a value for one invocation. A
`set` line in source applies afterward, so the file wins.
```bash
omnigraph query reports --query queries.gq --set merge_lineage=off --target ./graph.omni
omnigraph branch merge review/new-data --into main --set merge_lineage=verify --target ./graph.omni
```
`query`, `mutate` and `branch merge` apply settings to their operation.
`commit changes` and `changes poll` validate settings but they currently have
no effect. Direct `load` applies settings to its staged write; served `load`
refuses `--set` because its HTTP contract has no settings field.
A direct invocation accepts process settings from `--set`, source and the
environment. A server or managed invocation refuses process settings in a
request; those belong to the serving process. A stored query without `-e` or
`--query` takes no `--set` and uses process defaults.
## Read and write Blob values
```bash
omnigraph blob get node Document doc-42 body --target ./graph.omni --out body.bin
omnigraph blob stat node Document doc-42 body --target ./graph.omni --json
omnigraph blob put node Document doc-42 body --target ./graph.omni --file body.bin
```
`blob get` writes bytes to stdout when `--out` is omitted; `--offset` and
`--length` select a range. The CLI reports external references without
following them. `blob put` reads stdin when `--file` is omitted; `blob clear`
sets a nullable cell to null. Both accept `--if-match`. See
[Blob values](https://www.omnigraph.dev/docs/schema/blobs.md).
## Inspect and maintain a graph
```bash
omnigraph snapshot --target ./graph.omni --json
omnigraph commit list --target ./graph.omni --json
omnigraph schema show --target ./graph.omni
omnigraph optimize --target ./graph.omni
omnigraph cleanup --target ./graph.omni --keep 10 --older-than 7d
omnigraph cleanup --target ./graph.omni --keep 10 --older-than 7d --confirm
```
Maintenance uses direct storage. For a graph in a direct root, use
`--target ./cluster-root --graph knowledge`. Recovery effects that require an
explicit destination refuse a default-only selection. Read the
[maintenance guide](https://www.omnigraph.dev/docs/operations/maintenance.md) before cleanup.
## Use a server
Configure a server target with `auth: {provider: token, source: {store: {}}}` as shown
in [operator configuration](https://www.omnigraph.dev/docs/cli/targets.md), then install its token explicitly:
```bash
printf '%s\n' "$SERVER_TOKEN" | omnigraph credentials set --target standalone --stdin
omnigraph graphs list --target standalone
omnigraph query sources_for_claim --target standalone --graph knowledge \
--params '{"claim":"lower-latency"}'
omnigraph credentials status --target standalone --json
```
The server target's declared `auth` is the only credential source. There is no
implicit environment override or plaintext credential fallback. `login` and `logout`
manage platform sessions; static tokens use `credentials set/remove`.
The server resolves the actor from authentication; on a served write `--as`
only asserts it. Register remote endpoints rather than passing inline URLs.
### Use a server through a Unix socket
Operator version 1 does not accept `unix://` targets. Register an HTTPS server
endpoint instead; anonymous loopback HTTP is the only plain-HTTP exception.
Other clients with Unix socket support may use a
[TCP relay gateway](https://www.omnigraph.dev/docs/getting-started/deployment.md#unix-socket-access). This does not enable
Unix socket access in the current CLI.
## Manage a cluster
`--config DIR` selects a directory containing `cluster.yaml`; it defaults to
`.` for direct and server source operations. `.` means the current directory.
`--target` independently selects the direct root, server or managed cluster.
```bash
omnigraph cluster validate --config ./company-brain
omnigraph cluster plan --config ./company-brain --target ./cluster-root
omnigraph cluster apply --config ./company-brain --target ./cluster-root
omnigraph cluster status --target ./cluster-root
```
Validation is local and takes no target. Plan reads the selected destination
without acquiring writer admission. Apply owns exclusive admission for direct
storage or submits to the selected server. It manages graph definitions,
schemas, stored queries, policies, embedding providers and external Blob rules.
An optional `storage` in the bundle must agree with the target. See
[deployment boundaries](https://www.omnigraph.dev/docs/clusters.md#deployment-boundaries).
An explicit source path distinguishes a fresh direct apply with a caller-chosen
ID from source-free observation of that ID:
```bash
omnigraph cluster apply --target ./cluster-root --config . --deployment-id ID
omnigraph cluster apply --target ./cluster-root --deployment-id ID
```
Reusing an existing ID only observes its original invocation. It never executes
it again. Recovery that needs proof of stopped writers additionally requires
`--writers-stopped` and an explicit target.
For managed operations, register a profile and then a target using
service-attested cluster identity:
```bash
omnigraph login --create-profile work
omnigraph cluster list --profile work --json
omnigraph target add cloud --profile work --cluster CLUSTER_ID
omnigraph cluster plan --target cloud --json > plan.json
omnigraph cluster apply --target cloud --plan "$(jq -r .data.preview_id plan.json)" --json
omnigraph cluster status --target cloud --json
```
Managed planning uses pushed configuration. It takes no local `--config`;
`cluster push --config DIR` uploads the declared sources. See
[managed data access](https://www.omnigraph.dev/docs/cli/managed-data.md), [team administration](https://www.omnigraph.dev/docs/cli/managed-team.md)
and the [managed command reference](https://www.omnigraph.dev/docs/cli/reference.md#managed-cluster-commands).
## Output and errors
For `query` and `alias`, output selection follows `--json`, then `--format`,
then operator `defaults.output`, then `table`. Aliases have no format field.
Structured execution errors use the selected JSON, JSONL, or TOON format. Mutations
require explicit `--json` for structured errors. Target diagnostics go to
stderr and cannot be suppressed by a quiet flag.
Each invocation reads operator configuration once when needed. Edits cannot
change its destination or presentation defaults during execution. Unknown
fields, duplicate YAML keys and legacy routing sections are errors. Local
commands such as help, version, source validation and skill export do not load
operator configuration.
Output failures return nonzero without undoing completed work or automatically
repeating it. An accepted managed operation retains its recovery identity even
if its normal report cannot be delivered. See
[output formats](https://www.omnigraph.dev/docs/cli/reference.md#query-inputs-and-output) and
[diagnostics](https://www.omnigraph.dev/docs/queries-search/diagnostics.md).
## Give an agent the matching skill
```bash
omnigraph skill export ~/.claude/skills/omnigraph --force --json
omnigraph skill show references/queries.md
```
Running `omnigraph` or `omnigraph help` says to call that skill before a first schema or query. The binary carries the agent skill matching its commands and HTTP contract.
For a first setup, point the agent at the skill's onboarding guide
(`omnigraph skill show references/onboarding.md`): it goes from nothing
installed to a queried, branched and merged local graph.
Export refuses existing files unless `--force`; a later forced export removes
obsolete recorded files only while their bytes remain unchanged. Neither
command reads operator configuration or contacts a server.
# Managed data access
Source: https://www.omnigraph.dev/docs/cli/managed-data
This guide covers data credentials for an existing managed cluster. Complete
[managed login and cluster selection](https://www.omnigraph.dev/docs/cli/reference.md#managed-cluster-commands)
first; data authority is separate from that control-plane session.
After configuring a complete managed target and supplying its selected profile's
credential (login for an `omnigraph` profile), run:
```bash
omnigraph graphs list --target production --discovery
omnigraph query find_person --target production --graph knowledge --params '{"name":"Alice"}' --json
omnigraph mutate add_person --target production --graph knowledge --params '{"name":"Alice"}' --json
```
The target references a profile for API, organization and one auth provider, and
pins cluster ID, incarnation and data endpoint. See [operator configuration](https://www.omnigraph.dev/docs/cli/targets.md). A graph operation requires `--graph` or that target's
`default_graph`. Discovery lists graph IDs; it never selects one.
The CLI acquires a missing or expired data credential before submitting the
operation. Human profiles and stored-token profiles use the OS secure store.
Explicit env/file/exec profiles instead keep exchanged credentials only in the
current invocation's memory; they do not probe the OS store or select this path
after a store failure. Managed-target credential export follows the same rule.
External sources still require control-plane identity verification and exchange.
Authentication refresh never replays a submitted mutation.
`cluster token --target production --ttl 1h` is explicit persistent credential
administration and requires the OS secure store, even with an external source.
The issuer must support identity credentials and admit your principal to the
selected cluster. The credential proves who you are and which cluster you
can connect to; it contains no graph or action permissions. The cluster's
applied [Cedar policy](https://www.omnigraph.dev/docs/operations/policy.md) supplies those permissions.
For the example's stored queries, it must permit `invoke_query` plus `read`
or `change`. Ad-hoc query and mutation source uses `read` or `change`
respectively. Control-plane admin or apply permission does not confer graph
permission.
Apply graph policy grants, revocations and binding changes through the cluster
configuration workflow; editing a source file alone has no effect. Changes use the
[cluster configuration workflow](https://www.omnigraph.dev/docs/clusters.md#deploy-without-restarting);
identity credentials do not bypass its ownership or permission checks.
Issuance takes no `--graph`; choose a graph on the
operation that needs it. `--ttl` accepts seconds or an `s`, `m`, `h`, or `d`
suffix, defaults to one hour, and must be between 60 seconds and 24 hours.
The service can shorten the requested lifetime. Issuer clock tolerance is
30 seconds; a credential is never accepted after its stated expiry.
`--json` and human output contain metadata only, never the signed credential.
## Discover graphs
With a cached identity credential, `graphs list --target production --discovery`
shows every graph ID and display name in the server's applied inventory,
including graphs that failed to open. It does not reveal availability,
storage locations, schema, stored queries, or graph contents. A display name
currently equals its graph ID. Listing a graph does not grant access to it;
an absent policy or unknown policy actor still denies protected operations.
Use `--discovery` for this minimal inventory on either managed or standalone
server targets. Without it, `graphs list` requests graph metadata and requires
`graph_list` policy permission. The CLI does not guess the credential type from
its appearance. A server without discovery support, or an incompatible static
credential, refuses without falling back to another inventory.
## Cached access and routing
New human sessions can renew for 30 days from login. Refresh and repeated login
preserve saved deadlines, including older shorter sessions; provider inactivity
or revocation can end them sooner. Rotation stays in OS custody. Access JWTs
last at most 15 minutes, independently of graph credentials. Transient renewal
and identity-verification failures receive bounded retries; exhausted recovery
or a secure-store failure may still require explicit login. No graph operation
is replayed to repair authentication.
Use an upgraded CLI consistently for a profile. Older binaries reject a new
30-day cache; before deliberately downgrading, log out using the newer binary,
then sign in with the older one. Cache contents are never silently deleted.
Data credentials belong to the selected profile and resource binding,
including endpoint and incarnation. They remain separate from control sessions;
there is no plaintext cache or control-token fallback at the graph endpoint.
Use [credential export](https://www.omnigraph.dev/docs/cli/targets.md#manage-credentials) to explicitly release a
cluster-bound data token to another process through a pipe.
Managed and standalone access use the shared served graph client: query/mutation,
load, schema reads, access introspection, stored-query catalogs, branches,
snapshots, history, export, Blob operations, change feeds and the stdio MCP
bridge remain subject to server capabilities and policy. Direct-only
maintenance, schema changes and root administration do not acquire storage
access. `--as` remains refused on managed targets.
A valid data cache must match the captured endpoint, incarnation and identity.
A cached human sign-in or unchanged stored-token profile can support data access
during a control outage until expiry or signing-trust retirement. Replacing or
removing a stored token invalidates its cache generation. External env/file/exec
profiles require control verification and invocation-only exchange; do not assume
universal control-plane independence. Missing or expired access uses only the
selected provider; auth never falls back or replays an uncertain write.
Managed JSON requests refuse redirects and accept at most 8 MiB of response data,
with a 30-second deadline and at most 10 seconds to connect.
A lost write response never triggers an automatic retry. Managed loads
have the bounds below; Blob and streaming operations keep their own bounds.
Use `commit list --target production --branch main --graph knowledge --json`
and `commit show COMMIT_ID --target production --graph knowledge --json` to
inspect native lineage after an uncertain write. Commit identity, parents and
actor attribution are evidence; a timeout alone never proves that a mutation
failed to commit.
An explicit `--target staging` selects staging independently of source directories.
Configuration and cached credentials cannot silently reroute an invocation.
Preserve original recovery evidence when changing targets or credentials.
## macOS Keychain access failures
An upgrade can change the executable identity trusted by a Keychain item.
Ad-hoc signing does not provide a stable identity across builds. Stable release
trust requires consistent Developer ID signing and a fixed signing identifier;
an existing item approved for an ad-hoc build can still require authorization
when moving to a signed release. Several items can share the same service label,
so approving one item does not approve every session and data-cache entry.
Inspect the installed executable without reading credentials:
```bash
codesign -dv --verbose=4 "$(command -v omnigraph)"
codesign -d -r- "$(command -v omnigraph)"
```
On a secure-store failure, retain the operation, classified reason and numeric
OSStatus when present, together with `omnigraph version`. Do not export secrets
to diagnose it. An authorization failure alone does not prove that a password
is wrong, a keychain is locked, or a credential is missing. Do not repeatedly
approve prompts, delete or reset the keychain, allow all applications, or disable
signature checks as a workaround. Report repeated prompts to the maintainer.
See Apple's [code identity requirements](https://developer.apple.com/documentation/technotes/tn3127-inside-code-signing-requirements).
## Clear a credential
`cluster token --target production --clear` forgets that cluster's persistent local data
entry independently of its control session. Do not combine `--clear` with
`--graph` or `--ttl`. Clearing is not server revocation: copies remain usable
until expiry or signing-key retirement. `logout --profile work` clears the
selected local session and its derived data cache, then requests provider-session revocation;
`provider_revocation_confirmed` reports its outcome. Copies of issued data
credentials can remain valid remotely until expiry or signing-key retirement.
Local clearing does not require network transport. Logout also removes local
custody when the configured CA is unavailable or proxy selection fails, reporting
remote revocation as unconfirmed rather than retaining the session.
## Bulk loading
Use the ordinary `load` syntax with the selected target, an explicit
graph and mode. The native CLI sends the exact UTF-8 NDJSON body to the
authenticated data endpoint; it does not write storage directly.
```bash
omnigraph load --target production --graph knowledge --data batch-01.jsonl --mode append --branch review --from main --json
omnigraph load --target production --graph knowledge --data batch-02.jsonl --mode append --branch review --json
```
Every load requires `change` for that graph. `--from` additionally requires
`branch_create` when creating the target branch. Applied Cedar policy owns
target change permission and the base-to-target creation permission.
Identity credentials add no action ceiling. `invoke_query` is not required for loading.
Review and merge use the existing branch operations and their separate
permissions; a load does not merge its target branch.
One managed load accepts at most **32 MiB (33,554,432 bytes)** of input. The
CLI checks this before sending the request. Incremental `append` and `merge`
also retain the engine's **8,192 rows and 32 MiB per keyed table** bounds, plus
**32 MiB across retained keyed batches** and a separate parsed-payload estimate
across types. Strict loads check projected in-memory size too. The NDJSON byte
bound does not prove that a batch fits those
[engine limits](https://www.omnigraph.dev/docs/queries-search/mutations.md#limits-and-conflicts).
Split prepared `append` and `merge` inputs into suitable batches, preserving
endpoint dependencies and recording each batch's returned commit. Blob values carry
their own [limits](https://www.omnigraph.dev/docs/schema/blobs.md#limits).
`overwrite` also refuses when the IDs it removes exceed **32 MiB per
operation**, summed over all touched types. Each removed ID is charged its UTF-8
length plus 24 bytes, so the allowance holds 671,088 IDs of 26 bytes, the
length of a generated ID. Entities loaded without a `@key` and without an
explicit `id` get a new generated ID on every load, so overwriting them removes
every committed ID of that type. A delete and the edges it cascades to draw on
the same allowance. No setting changes it. An overwrite cannot be split; a
delete can be split into several commits.
The request has a **300-second deadline**, including receipt download, with
at most **10 seconds to connect** and **8 MiB of response data**. The CLI
never follows redirects or automatically retries the load. A timeout,
interrupted connection, response-limit failure, or invalid receipt can occur
after the server created a branch or committed data. Preserve the batch,
target and any response, then inspect that branch before deciding whether to
retry. `--from` is not an idempotency key. A success receipt's
`commit.actor_id` records the authenticated actor.
These managed bounds apply only to managed targets. Direct and server targets
use their own credential and transport contracts.
# Discover, configure and retire a managed cluster
Source: https://www.omnigraph.dev/docs/cli/managed-lifecycle
These commands use your managed service identity and permissions. They do not
open graph storage or require cloud credentials. The service must support
cluster creation and lifecycle operations; an older service returns a refusal.
Account commands select a profile and resource commands select a target.
Configure `work` as a managed profile and `production` as its target; see
[managed target configuration](https://www.omnigraph.dev/docs/cli/reference.md#managed-cluster-commands).
## Find a cluster and recover its configuration
After signing in, list the clusters visible to your selected organization:
```sh
omnigraph login --profile work
omnigraph cluster list --profile work --json
omnigraph target add production --profile work --cluster CLUSTER_ID
omnigraph cluster pull --target production --out customer-config --json
```
`pull` downloads the selected cluster's configuration into a **new** directory.
It does not write a folder binding or change the default target. Keep
`data.revision` for the next `push --expected-revision`; add `--rev FULL_COMMIT_ID`
to retrieve a historical source commit.
The destination's parent must exist. An existing directory, even empty, a file
or a symlink refuses. Download into another new directory to compare remote
changes with local edits; the CLI never merges or overwrites a working tree.
All source files appear together after validation. A failed download
leaves no partial destination. Source contents are not printed in the result.
Download is currently supported on Linux and macOS. The export is bounded to
4096 UTF-8 files, 1 MiB per file, 8 MiB total file contents, 512-byte relative
paths and a 64 MiB JSON response (allowing JSON escaping). Its HTTP deadline is
120 seconds. Traversal, symlinks, conflicting paths and reserved directories
are refused. `.git`, credentials and server-side control records are not
downloaded, and no local routing context is created.
The service requires active membership and organization-wide `source_read`
permission from WorkOS. An external repository binding must be checked out
through its own Git provider instead.
## List cluster members
Use the selected cluster target to list active organization members:
```sh
omnigraph cluster members list --target production
omnigraph cluster members list --target production --json
```
The command requires the organization's current `status` permission. It shows
provider user and membership IDs, available stable principal IDs, identity
eligibility and organization-wide service permissions. Names and emails may be
unavailable. A null principal means no stable identity mapping exists yet; it
is not a missing enrollment requirement. `--json` returns the API envelope,
including the provider observation time and expiry.
The list derives from the cluster's owning organization, with a limit of 1,000
provider memberships. An overflow or incomplete provider read refuses instead
of truncating. This read-only command does not calculate effective graph
permissions: applied Cedar policy authorizes those separately. Organization
administrators use [team administration](https://www.omnigraph.dev/docs/cli/managed-team.md) to invite and remove
people and WorkOS to assign service roles.
## Create an empty cluster
Use a complete profile:
```sh
omnigraph login --profile work
omnigraph cluster create customer-demo --profile work --no-wait --json
```
The service reserves a cluster and operation. Creation returns its identity;
it never creates configuration files, binds a directory or changes the selected
target. Keep `data.canonical_root`, `data.incarnation` and `data.operation_id`,
then register it with `target add production --profile work --cluster CLUSTER_ID`. Without `--no-wait`, the
command waits for initial bootstrap and serving readiness. `data.state` is the
operation outcome; `data.lifecycle.phase` reports provisioning progress.
`--no-wait` returns as soon as the operation is accepted. `--timeout SECONDS`
sets a local 1–3600-second deadline; the default is 300. A timeout does not
cancel creation. Follow the saved operation explicitly:
```sh
omnigraph cluster operation OPERATION_ID --profile work --json
omnigraph cluster operation OPERATION_ID --profile work --wait --timeout 600 --json
```
Profiles can observe creation before a cluster target exists. A cluster
target can observe only an operation for its pinned cluster and incarnation.
A `ready` cluster does not imply a reachable endpoint. The selected service must
attest the route; this configuration rewrite does not qualify private/BYOC routes
or tunnel transports.
## Prepare and apply configuration
Creation returns identity without creating local configuration files. After
the operation reaches `ready`, download its initial configuration, including
the policy that grants its creator management access:
```sh
omnigraph cluster pull --target production --out customer-config --json
# Keep data.revision as COMMIT below.
```
Edit `customer-config/cluster.yaml` to add a graph under `graphs`, keeping its
existing storage root and policy declarations:
```yaml
graphs:
catalog:
schema: ./catalog.pg
```
Create `customer-config/catalog.pg` with your graph schema, then submit the edit
against the exact revision downloaded:
```sh
omnigraph cluster push --target production --config customer-config --expected-revision COMMIT \
--message "Add the knowledge graph" --json
# Use data.revision returned by push.
omnigraph cluster plan --target production --rev NEW_COMMIT --json
# Use data.preview_id from the ready preview.
omnigraph cluster apply --target production --plan PREVIEW_ID --json
```
`plan` waits for the accepted preview by reading its original ID. `--no-wait`
returns the accepted record immediately; `--timeout SECONDS` bounds only the
local wait (300 seconds by default). A timeout or disconnect does not cancel
the preview or submit another one. Progress goes to stderr; stdout contains
the final response, including the native plan when ready.
The `plan` and `apply` submissions also stay open until that deadline, so a
service that captures the preview before answering is not cut off. Each
source lookup and poll keeps its own 10-second bound.
To resume waiting, repeat `plan` with the same `--rev` and
`--idempotency-key` printed by the first invocation. The original preview's
deadline does not change. A `failed` or `expired` preview requires a fresh
plan with a new key; its `failure_code` explains the terminal outcome.
`push` uploads only `cluster.yaml`, referenced schemas, stored queries and
policy files. Query-directory declarations discover immediate `.gq` files;
they do not recurse. Unrelated files, folder context and credentials are not
uploaded. Paths must stay within the config directory, with no symlinks or
special files. Files must be UTF-8, at most 2 MiB each, with at most 4096 files
and a 32 MiB request. Path length is limited to 512 bytes and capture to
120 seconds. Managed upload currently requires a Unix platform with
descriptor-relative file traversal; other platforms refuse before submission.
The expected revision is a full 40-character lowercase Git commit ID. A stale
head is refused; pull into another new directory and reconcile your edit before
retrying against that revision. The storage root cannot change. `push` requires
`source_write` and changes only the managed repository. Source-head observation requires
`source_read`; it grants no native management permission. The applied cluster
policy decides whether the caller may plan and deploy configuration. A preview
does not hold a change lease or prevent another source edit.
## Delete and undo
Use the exact incarnation from your cluster's response. `--incarnation` must
equal the configured target and cannot redirect deletion:
```sh
omnigraph cluster delete --target production --incarnation INCARNATION \
--idempotency-key customer-demo-delete --json
```
Delete needs current organization-wide `delete` permission. It closes new
deployment and data-token issuance; the service then withdraws routing and verifies quiescence. The
default undo interval is 86,400 seconds. With a nonzero interval, the command
returns when the service confirms `tombstoned`, including its deadline and
operation ID. Canonical `data.state` remains `running` until the service's
cleanup finishes; a tombstone does not mean purged.
`--retention-seconds` accepts 0–2,592,000. Zero requests immediate retirement
after quiescence and waits toward final cleanup within the local timeout.
`--no-wait` always returns the accepted operation without polling.
Before the deadline and irreversible retirement, request undo of the exact
deletion operation:
```sh
omnigraph cluster undo-delete --target production --incarnation INCARNATION \
--deletion-id DELETE_OPERATION_ID --json
```
Undo needs current organization-wide `delete` permission. It preserves the
cluster identity and restores the retained service; it does not authorize a new
configuration deployment. The service decides
whether the retained window is still open; the CLI never clears retirement or
creates a replacement. The configured target remains available after deletion for status and history.
## Recover an uncertain response
Lifecycle commands save their origin, exact request digest and idempotency key
with the authenticated principal and account before sending. Rerun the same
command as the same principal using the same operator home and target after a network
failure; its pending key is reused. A different request or conflicting explicit
key refuses rather than guessing whether a prior operation was accepted.
`--idempotency-key KEY` lets automation supply its own stable identity.
Renewing the same principal's session is supported; switching principals or
accounts refuses the unresolved retry.
Timeouts and unknown gateway responses keep the pending record, as does a
permission refusal on a retry after an uncertain response. A recognized
definitive refusal on the first attempt releases it so the request can be
corrected.
After validated acceptance, the operator home's `lifecycle/` directory retains
the exact request, key, operation and cluster identity before clearing the
pending record. Records are partitioned by API and organization, contain no
credential, and grant no permission. History is bounded to 128 accepted requests
per account and 1 MiB per file; reaching a limit refuses instead of discarding
recovery evidence. A completed request without an explicit key is not silently
submitted again. Use `cluster operation ID` to observe; supplying the retained
key explicitly resubmits only that same idempotent request.
Preserve legacy pending and accepted lifecycle evidence. Use the explicit
[operation-evidence recovery command](https://www.omnigraph.dev/docs/cli/targets.md#upgrade-from-the-old-format)
for old folder records; it does not convert config or submit operations. Resume
with the original verified principal and original idempotency key. Never delete
uncertain evidence to force a new submission.
JSON output is one response on stdout; progress and idempotency information use
stderr. A local deadline exits 5 and retains the latest operation response.
Lifecycle outcomes keep their own mapping: ready or purged exits 0, failed 1,
refused or blocked 2, partially converged 3, recovery required 4, stalled or a
local deadline 5, and cancelled 6. A retained tombstone is an observation goal
for deletion with a nonzero undo interval, not completed physical cleanup.
Lifecycle commands refuse `--as` and unsupported target kinds. Authentication and data credentials remain separate.
# Onboard and remove organization members
Source: https://www.omnigraph.dev/docs/cli/managed-team
Invite teammates and manage organization membership through the managed CLI.
The identity provider owns membership and organization-wide service permissions;
the applied cluster configuration owns graph, branch, schema and deployment
permissions.
No database or cloud credentials are needed. The managed service must support
organization administration, and an administrator must configure its provider
membership and service roles.
Sign in to the selected organization:
```sh
omnigraph login --profile work
```
Every command below uses `--profile work`, a managed profile with an explicit
API, organization ID and authentication provider. See
[operator registration](https://www.omnigraph.dev/docs/cli/targets.md#login-and-registration). `org` aliases
`organization`. The caller needs the actual current `members_manage`
permission. Role names and permission claims in an older bearer do not confer
service authority. Role definitions and their service permissions remain owned
by the identity provider.
These commands never use folder context or change the selected target. A
cluster target cannot administer its organization implicitly. `--json` emits
the validated API envelope.
## Invite and find people
```sh
omnigraph organization --profile work invitations send avery@example.com \
--role member --idempotency-key invite-avery-20261015
omnigraph organization --profile work invitations list --json
omnigraph organization members list --profile work --json
```
Choose an existing service-role slug for every new invitation; there is no
implicit default. The invitation email lets the recipient join the organization. After
acceptance, active membership establishes identity eligibility for its
clusters. There is no additional local enrollment step. Graph access still
requires permission in the applied cluster policy.
The complete organization directory shows names/emails, active, pending and
inactive membership states, assigned service roles, service permissions and
available `principal:ID` values for Cedar. A principal can be absent until the
person's first sign-in. `--json` also includes exact user and membership IDs
and the provider observation time and expiry. More than
1,000 memberships or an incomplete provider read refuses the whole directory.
Invitation lists are separate pages of up to 100. Pass the returned `data.after`
as `--after CURSOR` to fetch the next page. A null cursor means the last page.
To resend or cancel an invitation, use the recipient's email or its exact ID:
```sh
omnigraph org invitations resend avery@example.com --profile work \
--idempotency-key resend-avery-20261015
omnigraph org invitations revoke avery@example.com --profile work \
--idempotency-key revoke-avery-20261015
```
An email must match exactly one pending invitation in this organization. The
CLI scans at most ten pages and 1,000 invitations within 35 seconds, refusing
incomplete or ambiguous results. Use an exact invitation ID to address a
particular invitation. Output never includes acceptance URLs or invitation
tokens.
## Assign permissions and connect
Assign an existing active member one or more configured service roles:
```sh
omnigraph org members set avery@example.com --role member --profile work \
--idempotency-key role-avery-20261027
omnigraph org members list --profile work
```
Replace `member` with a role slug configured for your organization. `members
list` shows current assignments and their permissions, not every available
role. Repeat `--role` to select multiple roles. The command replaces the complete
assigned set, so include every role the member should retain. Duplicate flags
are refused; changing their order does not change the request. Use an exact
user ID instead of email when needed. The command does not create a person,
membership or role.
Directory-managed memberships must be changed through their directory.
Service roles contain these organization-wide permissions:
| Permission | Scope |
| ---------------- | ----------------------------------------------------------------- |
| `members_manage` | Organization directory, invitations, role changes and offboarding |
| `create` | Create clusters in the organization |
| `status` | Discover and inspect the organization's clusters |
| `delete` | Delete or undo deletion of any cluster in the organization |
| `source_read` | Read configuration source for the organization's clusters |
| `source_write` | Update configuration source for the organization's clusters |
These permissions apply across the organization. They do not authorize native
configuration deployment or graph operations. `organization members apply`
and local grant patches are no longer supported.
The teammate registers their profile and signs in. An administrator can then
read their stable `principal_id` using `members list`. A configuration manager
adds `principal:PRINCIPAL_ID` to the appropriate Cedar policy through the
[source and deployment workflow](https://www.omnigraph.dev/docs/cli/managed-lifecycle.md#prepare-and-apply-configuration).
Grant only the intended graph and configuration permissions, retaining another
manager.
With `status` permission, the teammate discovers a cluster and registers its
service-attested identity:
```sh
omnigraph login --create-profile work
omnigraph cluster list --profile work
omnigraph target add production --profile work --cluster CLUSTER_ID
omnigraph target inspect production --resolved
```
Verify a permitted graph read, a reversible write if granted, and a forbidden
operation. Cluster discovery and identity-credential issuance alone do not
prove graph permission. Service checks read current provider membership and
permissions; provider-side role propagation can take time.
## Remove access
First remove the person's stable principal from applicable graph and management
Cedar policies, deploy through the normal configuration workflow, and verify
denial using a previously issued graph credential. Retain another authorized
configuration manager.
Then offboard the person by email or exact user ID:
```sh
omnigraph organization members offboard avery@example.com --profile work \
--idempotency-key offboard-avery-20261015 --json
```
The CLI resolves and retains the exact current membership before sending.
Email must match exactly one member in this organization. For a scripted
assertion, add `--expected-membership om_example`; it must match the observed
membership. Use an exact user ID with `--expected-membership absent` only when
membership is already absent and you need to finish session cleanup. A
different or recreated membership refuses; inspect the directory before
starting a new request. The service checks that another active member retains
`members_manage` before removing the last observed manager. This check does not
lock concurrent changes made through the identity provider.
Offboarding removes the selected organization's membership and revokes sessions
explicitly belonging to that organization and person. It preserves the shared
provider user and other organizations' sessions. The receipt reports provider
membership and session cleanup separately. A partial result exits 3; complete
cleanup exits 0. It never claims to change graph policy. Existing graph
credentials can last up to 86,400 seconds, so membership removal alone is not
immediate graph revocation.
## Recover an interrupted request
Every write requires an explicit `--idempotency-key`. Before sending, the CLI
durably retains the exact resolved request in private `OMNIGRAPH_HOME` custody.
Keep this home, the same profile/identity, arguments and key until the outcome
is known. Repeating the identical command uses the retained request; it does
not resolve the email again. Changed arguments or a different identity refuse.
Custody is bounded to 128 records and 1 MiB per stable profile identity, with no
automatic eviction. At the limit, new writes refuse while exact retries remain
available.
The CLI submits once with a 35-second request deadline. A timeout does not
cancel a provider action. Repeat the identical command with the same key to
retrieve its retained result. The service never resends an uncertain provider
write. A new key starts a new action, so do not change it merely to retry.
A role-change receipt distinguishes a verified update from an uncertain
provider outcome. An uncertain result exits 3 and shows the last verified
roles and permissions; it does not claim that the requested role took effect.
If an older CLI saved an invitation without a role, repeat its original
arguments and key to retrieve the saved result or uncertainty. The CLI preserves
that request unchanged; a new invitation always requires `--role`.
A partial offboard result stays partial on replay. Inspect the current provider
state and retained receipt before preparing a new request for remaining cleanup.
An uncertain offboard blocks other offboards in that organization until the
service operator reconciles the retained attempt; a new key cannot bypass it.
Human progress and the key go to stderr; `--json` keeps one response on stdout.
Permission refusals exit 2 and malformed/transport failures exit 1, with no
storage fallback.
## Configure an unattended workload profile
The managed service must support workload sessions. Ask its administrator to
configure an organization-owned client application and its service permissions.
Obtain and rotate its secret through your identity provider's administration
tools. Add a profile to
`~/.omnigraph/config.yaml` (or `$OMNIGRAPH_HOME/config.yaml`):
```yaml
version: 1
profiles:
ops:
api: https://control.example.com
organization_id: org_example
auth:
provider: oauth_client_credentials
token_endpoint: https://auth.example.com/oauth2/token
client_id: client_example
client_secret: {env: OG_OPS_CLIENT_SECRET}
# Optional provider scope, not graph permissions:
scope: "read write"
```
The secret may instead use `{file: /var/run/secrets/ops-client-secret}`.
Environment values are exact; files allow one final LF or CRLF and atomic
symlink rotation. Sources are bounded to 64 KiB. Literal YAML secrets and
secure-store client secrets are unsupported.
```sh
omnigraph cluster list --profile ops
omnigraph target add ops-cluster --profile ops --cluster CLUSTER_ID
omnigraph --target ops-cluster --graph knowledge query -e 'query q() { return { 1 as ok } }'
omnigraph mcp --target ops-cluster
```
Acquisition makes one form-encoded `client_credentials` request using
`client_secret_post` at the explicit HTTPS endpoint. It uses system trust,
direct networking and a 25-second total bound, without endpoint discovery,
redirects, browser login or refresh tokens. The verified service session must
identify the configured client as a verified workload in the selected
organization. Provider service permissions govern organization operations;
applied Cedar governs graph and deployment operations. Provider scopes do not
replace either authority.
The token response must include a positive `expires_in`; providers that omit it
are unsupported. The profile's proxy and custom-CA settings apply to managed
service requests, not this token exchange.
Tokens stay in invocation memory. Long-running commands renew expired control
credentials before their next request and keep the same verified organization,
principal, kind, subject, client ID and assurance. A typed native-credential
issuance preflight refusal may trigger one renewal and one new issuance request. A
submitted operation, HTTP 401 or uncertain delivery is never replayed for
authentication. Secret rotation takes effect on the next acquisition; a
different client or identity needs a separate profile.
OAuth `cluster token` reports issuance metadata without persisting the credential;
in a fresh invocation, `--clear` reports `local_credential_removed: false` because
there is no persisted OAuth credential to remove. It does not clear credentials
saved by a previous provider. Ordinary resource commands acquire their own
credential.
# CLI reference
Source: https://www.omnigraph.dev/docs/cli/reference
Use `omnigraph --help` or `omnigraph --help` for exact installed flags.
## Addressing a graph
Resource commands resolve one target from `--target NAME|LOCAL_PATH` or
`defaults.target`. Account commands use `--profile NAME` or
`defaults.profile`; see [selection rules](https://www.omnigraph.dev/docs/cli/targets.md#selection-and-defaults). An unknown, incomplete or unsupported
target refuses; it never falls back to another backend. Folder context and
source configuration do not select destinations. Every backend invocation prints
its resolved target to stderr, including its kind, scope and selection source.
Machine results remain on stdout.
| Target | Example |
| ----------------------------- | ----------------------------------------------------------------------- |
| Registered direct graph | `omnigraph query --target local -e 'branch list'` |
| Inline direct graph | `omnigraph query --target ./graph.omni -e 'branch list'` |
| Direct cluster root and graph | `omnigraph query --target ./cluster --graph knowledge -e 'branch list'` |
| Server | `omnigraph query --target standalone --graph knowledge find_person` |
| Managed cluster | `omnigraph query --target production --graph knowledge find_person` |
A registered direct target declares `scope: graph` or `scope: root`; a root or
server requires `--graph` or its own `default_graph` for graph operations. No
request discovers a graph to select automatically. Inline addresses are local paths or `file://` URIs, plus
[S3 cluster roots](https://www.omnigraph.dev/docs/operations/s3-cluster-roots.md) for deployment, recovery and maintenance commands. Register remote
endpoints; other cloud-direct and catalogue targets are unsupported. Use `./name` for a
relative filesystem address; a bare name selects a registered target.
`init --schema graph.pg ./graph.omni` remains a creation convenience and refuses `--target`.
`--base URI` opens a direct graph target as a graph shallow clone of the graph
at `URI`: a read falls through to the base where the target has no object of its
own, every write lands in the target, and the base is never written. Both are
local directories or `s3://` prefixes of one endpoint (an S3 pair reads the
standard `AWS_*` environment, and the clone's `s3://` prefix is named inline as
the target, the one inline `s3://` graph the CLI admits); a target that contains
its base, or lies inside it, is refused. The option applies to `query`, `mutate`, `load`, `branch`,
`commit`, `changes`, `diff`, `snapshot`, `export`, `blob` and `schema show`;
maintenance and schema changes keep the one root they are given. A fresh target
directory or prefix is a new clone; the same target again shows the clone's own
state. Nothing in the target records its base, so pass the same `--base` on
every open of that target; a clone root opened without `--base` is refused,
since its marker says it is a clone and not which base it was cloned from.
A target that already holds objects but no clone marker, an ordinary graph
for example, is refused under `--base` before anything is written.
`--as ACTOR` attributes supported direct operations; on a cluster graph the
applied graph policy judges them by it ([direct writes](https://www.omnigraph.dev/docs/operations/policy.md#actor-identity)).
A served target derives its
actor from its credential and sends `--as` as the `Omnigraph-If-Actor` assertion,
exiting 4 on a mismatch ([Assert the actor](https://www.omnigraph.dev/docs/operations/server.md#assert-the-actor));
a managed target refuses it. `--yes` supplies noninteractive consent for destructive
operations on nonlocal storage. The retired `--quiet` cannot hide the destination.
## Commands
| Command | Purpose | Scope |
| ----------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------- |
| `init` | Create an empty graph from a `.pg` schema | direct |
| `catalogue` | Query this binary's engine registries with `-e` or `--query`; optional `--json` | local, no operator config or graph storage |
| `query` | Run a read query, or a branch read (`branch list`, `diff`, `merge-base`, `merge … dry run`), `show`, or `explain` statement | direct or served |
| `mutate` | Run an insert/update/delete query (`as commit` leaves it unpublished; `as unattached branch [on ""]` records a snapshot that never publishes), or a `branch create`, `branch delete`, `branch merge`, `commit publish`, `commit drop`, or [`load csv`](https://www.omnigraph.dev/docs/queries-search/mutations.md#csv-loading) statement | direct or served |
| `load` | Load graph JSONL in `overwrite`, `append`, or `merge` mode | direct or served |
| `blob get/stat/put/clear` | Read, inspect, replace or clear one Blob cell | direct or served |
| `branch create/list/delete/merge/diff/merge-base` | Manage graph branches; `create --from-commit` starts a branch at a commit, `list --detailed` adds each branch's head, start and times, `merge --as-commit` leaves the merge unpublished, `merge --dry-run` reports the outcome or conflicts and writes nothing, `diff` reviews what a branch changed since its merge base, `merge-base` prints that base | direct or served |
| `snapshot` | Show a branch snapshot: `internal_schema_version`, `graph_manifest_version`, and the datasets of one captured graph version | direct or served |
| `commit list/show/changes/publish/drop` | Inspect history (`list --limit`/`--page-token` for one page of it, `list --unpublished` for a branch's unpublished commits, `list --unattached` for its live unattached snapshots; the two flags exclude each other and paging) or one commit's entity changes; publish an unpublished commit onto its recorded target branch (an unattached snapshot refuses with `unattached_cannot_publish`), or drop it (an unattached snapshot takes every descendant with it); `publish` and `drop` take `--as`, the reads refuse it | direct or served |
| `diff` | Net entity changes between two points: branches or commit ids (`--from`, `--to`) | direct or served |
| `changes poll/baseline` | Consume a branch change feed or establish a new baseline | direct or served |
| `export` | Stream a branch as JSONL | direct or served |
| `schema show` | Read the accepted schema | direct or served |
| `schema apply` | Apply a schema to a standalone graph | direct |
| `schema plan` | Preview a schema migration | direct |
| `schema upgrade-system-columns` | Respell a graph's system columns in place; needs a graph that normal open accepts and keeps its storage format | direct |
| `lint` | Validate `.gq` source | local schema or direct graph |
| `upgrade` | Convert a v8, v9 or v13 graph's storage to the format this binary serves, offline; `--check` writes nothing | direct standalone |
| `optimize` | Compact data and reconcile declared indexes on `main`; a server target runs it as a [recorded operation](https://www.omnigraph.dev/docs/operations/recorded-operations.md), waiting for its record unless `--no-wait` | direct or served |
| `index build` | Build a branch's missing declared indexes and refresh lagging BTREE and vector coverage ([Maintenance](https://www.omnigraph.dev/docs/operations/maintenance.md#build-indexes)) | direct |
| `index rebuild --kind full-text` | Replace full-text indexes on one branch; `rebuild-full-text-indexes` is its deprecated spelling | direct |
| `cleanup` | Delete table versions that no retained graph commit pins, under an explicit retention policy ([Maintenance](https://www.omnigraph.dev/docs/operations/maintenance.md#cleanup)) | direct |
| `graphs list` | List graph metadata or minimal identity discovery | served |
| `queries list` | List stored queries: a cluster root's registry (`--graph` narrows it to one graph), or one served graph's catalog, whose `--json` is the body of `GET /graphs/{id}/queries` | cluster or served |
| `queries validate` | Type-check a cluster's applied query registry | cluster |
| `cluster validate/plan/apply/...` | Operate self-hosted declarative state; [deployment/recovery flags](https://www.omnigraph.dev/docs/clusters.md) | config, server, explicit root |
| `cluster create/list/members/pull/push/delete/undo-delete/token/operation/history/cancel` | Operate a managed cluster | profile or managed target |
| `policy validate/test/explain` | Validate or evaluate applied policy | cluster |
| `embed` | Generate, clean, or refresh seed embeddings | local tooling |
| `login`, `logout` | Create a profile with `login --create-profile NAME` or manage an existing `omnigraph` profile session | managed profile |
| `target list/inspect/add/refresh/remove` | Inspect/edit targets; managed add/refresh uses discovery | local or managed API |
| `credentials set/status/remove/export` | Manage static custody or export access to a pipe; explicit profile or standalone target required | profile or server; managed target for export |
| `config validate` | Validate the whole operator file offline | local |
| `config rename-profile/rename-target/recover-rename` | Rename local bindings while preserving custody, or finish an interrupted rename ([syntax](https://www.omnigraph.dev/docs/cli/targets.md#rename-registrations-and-profiles)) | local |
| `config recover-operations --source DIR` | Preserve legacy folder lifecycle evidence; no operation submission | profile, local evidence |
| `access` | Say who you are at a target and which actions its policy grants, graph-wide and on a branch ([access](https://www.omnigraph.dev/docs/cli/targets.md#what-you-may-do-at-a-target)) | direct, server or managed |
| `mcp` | Relay an agent's stdio MCP to the target's `/mcp` with its resource credential, bound to one graph; write tools need `--write` ([stdio agents](https://www.omnigraph.dev/docs/operations/mcp.md#connect-a-stdio-agent)) | server or managed |
| `studio` | Open Omnigraph Studio on a server target through an `omnigraph-studio://open` link and print it; no request or credential; `--json` adds the `binding` and `home` digests ([Open Studio](https://www.omnigraph.dev/docs/cli/studio.md)) | server |
| `alias` | Invoke a personal stored-query alias bound to one target | served |
| `organization members/invitations` | List members, set service roles, offboard and manage invitations; see [team access](https://www.omnigraph.dev/docs/cli/managed-team.md) | profile |
| `skill export/show` | Export the agent skill built into this binary, or print one of its files ([matching skill](https://www.omnigraph.dev/docs/cli.md#give-an-agent-the-matching-skill)) | local |
| `version` | Print the package version, HTTP contract and storage formats; `--json` includes `version`, `http_contract`, `protocols` (`studio: 1`, `target: 2`, `credential: 2`, `access: 2`) and `internal_schema` (`writes`, `serves`) ([Verify](https://www.omnigraph.dev/docs/getting-started/install.md#verify)) | local |
## Query inputs and output
For ad-hoc source, pass `--query ` or `-e/--query-string `; with
multiple declarations the positional name selects one. A stored server query is
its registry name alone. Parameters come inline, `--params '{"name":"Ada"}'`,
or from a file, `--params-file params.json`. `--set NAME=VALUE`, repeatable,
gives a [session setting](https://www.omnigraph.dev/docs/cli.md#session-settings) a value for the run of
`query`, `mutate`, `branch merge`, `commit changes`, `changes poll`, and `load`.
The source may instead be one branch statement, `mutate -e
'branch create b0'` or `query -e 'branch list'` (writes through `mutate`, the
listing through `query`; no `--branch`, `--snapshot`, `--if-commit`, name, or
params; see [Work with branches](https://www.omnigraph.dev/docs/cli.md#work-with-branches)), or one
`explain` statement, `query -e 'explain query q() { … }'`, answering the plan
as rows under the query's own target and params; see [Explain](https://www.omnigraph.dev/docs/queries-search/explain.md).
Read output supports `table`, `json`, `jsonl`, `csv`, `kv`, `toon`, and, for
`query`, `arrow`. With no
`--format`, no `--json`, and no `defaults.output`, a read prints a table. `--json` is the
stable machine-readable form for commands that do not use `--format`. Result
cells use the [JSON result spelling](https://www.omnigraph.dev/docs/queries-search/queries.md#json-result-spelling);
`table` and `kv` print strings unquoted. CSV writes a null as an empty field
and an empty string as `""`, and quotes fields containing a comma, double
quote, or line break, so [`load csv`](https://www.omnigraph.dev/docs/queries-search/mutations.md#csv-loading)
reads a null back as null and the string `null` as that string. `load csv`
also reads `""` as null, because it has no spelling for an empty string, and
it skips empty lines, so a null row of a one-column result is not loaded back.
`--format json` prints the
envelope pretty and the `rows` array compact. `--format toon` encodes that same
document as [TOON](https://github.com/toon-format/spec) (specification 4.4,
comma delimiter, two-space indent). `rows` declares its fields once when every
row has the same keys. A null cell omits its key, so one null makes the rows
non-uniform and TOON repeats every key. Numbers use TOON's canonical spelling,
so an integral value is `1` where the JSON writer may spell `1.0`. The CLI ends
the TOON document with a newline. The HTTP API returns JSON. A refusal follows
[Diagnostics](https://www.omnigraph.dev/docs/queries-search/diagnostics.md).
`query --format arrow` writes the result's columns and rows to stdout as one
[Arrow IPC stream](https://arrow.apache.org/docs/format/Columnar.html#ipc-streaming-format),
which Arrow readers such as pyarrow and Polars read. Each column keeps its
declared Arrow type, and a null stays a null. The stream holds the result
alone: the `graph_commit_id`, target and `ranking` of the JSON document are not
in it. Only a query that this process runs on a direct target, including an
`explain` statement, has result batches to write. Three cases refuse `arrow`
before the read runs, and exit 1: a served target, whose server answers JSON
(refused before its credential is read); a control statement such as
`branch list` or `show`; and a stdout that is a terminal, so redirect it, for
example `> result.arrow`. Stdout therefore holds the whole stream, or, when a
write to it fails, part of one and a nonzero exit. `alias` and
`defaults.output` do not offer `arrow`. To stream a whole branch, use
[`export`](#commands), which writes JSONL.
### Invocation statistics
`--stats PATH` writes one JSON document when the invocation ends, a refusal
before the command dispatches included: `stats_version` (`1`), `elapsed_ms`
(the invocation from target resolution to its last byte of output), `exit_code`
(the code the process exits with), `storage` with the logical object-store
calls the engine issued on the `manifest` and `table` planes of a direct target
(`get`, `put`, `put_part`, `head`, `list`, `delete`, `copy`, `rename`,
`multipart_complete`, `multipart_abort`; every direct target counts, a local
directory included; a served target issues none in this process, so both planes
read zero there), and `peak_rss_bytes` where the platform reports it. One
logical call may fan out into several requests below the engine, so these are
never request or cost counts. The document is the whole invocation, not one
statement, and a document that cannot be written fails the command after it
ran, so a write it carried has committed.
### Machine-readable read and write positions
`query --json` and `query --format toon` return `graph_commit_id` when the read snapshot has a graph head. The id and
rows share one pinned snapshot; use that id for a later conditional mutation. A ranked read also
returns `ranking` on that document, and in the `--format jsonl` metadata line too
([what a ranked read reports](https://www.omnigraph.dev/docs/queries-search/search.md#what-a-ranked-read-reports)).
Successful `mutate --json` and `load --json` responses include `commit`, the exact commit published by
that attempt: `graph_commit_id`, optional `graph_branch`, `graph_manifest_version`, optional parent and
merged-parent ids, optional `actor_id`, and `created_at` in Unix microseconds. A successful mutation
that changes no entities returns `"commit": null`. On every target `load --json` prints the body of
`POST /graphs/{id}/load/ndjson`, with the write's `actor_id` and a `base_branch` that is `null` without `--from`.
`--json`, read `--format json`, and read `--format toon` preserve structured errors on stdout. `--format toon` encodes the same error document as `--format json`. Data-write and read failures
report [whole-command outcomes and exits](https://www.omnigraph.dev/docs/operations/troubleshooting.md#failed-data-write-commands); a refused command line exits 64.
Verified conditional mismatches exit 4, embedded or served, when nothing else in the command ran.
### Conditional mutations
```bash
omnigraph query find_person --query queries.gq --target ./graph.omni --json
omnigraph mutate update_person --query queries.gq --target ./graph.omni \
--if-commit --json
```
`--if-commit` runs the mutation only while the target branch is still at that
commit. Any intervening commit on the branch invalidates the condition, even
when it changed unrelated data. A mismatch has no effect and exits with code
4, against a server or embedded; JSON output includes `precondition_failure` with `expected` and optional
`actual` commit ids. Re-read and decide again instead of retrying blindly.
## Storage upgrade
```bash
omnigraph upgrade --target ./graph.omni --check --json
omnigraph upgrade --target ./graph.omni --json
omnigraph schema upgrade-system-columns --target ./graph.omni --check --json
```
`omnigraph upgrade --target ADDRESS`
converts a standalone v8, v9 (release 0.11.x) or v13 graph to v17, offline and
in place, keeping branches and commit history, and stamps a v14, v15 or v16 graph
to v17 with one metadata-only commit per live branch; stop every process using
the graph and retain a verified whole-root backup first. `--check` writes
nothing; `--to-format` accepts 17 only. `check_passed`, `already_current` (a
v17 graph) and `completed` exit 0; `check_failed` (`unsupported_source` for any
other format below 14, `newer_than_binary` above 17, `unsupported_target`)
and `recovery_required` (a pending attempt, finished as `recovery.action` says,
or leftover recovery files) exit 1. The report names the formats, the route,
the `work` counts, findings and the recovery action. Server and cluster
addressing are refused; see [storage upgrade](https://www.omnigraph.dev/docs/operations/upgrade.md#storage-upgrade).
`schema upgrade-system-columns` is a separate operation on a supported standalone graph: [system-column upgrade](https://www.omnigraph.dev/docs/operations/upgrade.md#system-column-upgrade-legacy-spellings).
## Load modes
`load --mode` is required:
| Mode | Existing entities | Typical use |
| ----------- | ------------------------------------------------------------------------------- | ------------------------------------------- |
| `overwrite` | Each node or edge type represented in the batch is replaced; other types remain | Initial load or import of a complete export |
| `append` | Kept; duplicate IDs fail | Strict batch insertion |
| `merge` | Updated by ID | Idempotent synchronization |
`--branch ` selects an existing branch. Add `--from ` to create a
missing branch from an explicit base. Overwrite is destructive and may require
`--yes` for non-local storage.
For a managed cluster target, `load --target NAME --graph ` uses the separate
cached data credential. It requires `change`, plus `branch_create` when
`--from` is present. Managed loads bound input to 32 MiB, responses to 8 MiB,
and one request to 300 seconds; uncertain writes are never automatically
retried. See [managed bulk loading](https://www.omnigraph.dev/docs/cli/managed-data.md#bulk-loading) for limits,
permissions, ordinary addressing, and reconciliation.
Diffs, branch review and change feeds: [Changes, Diffs and Change Feeds](https://www.omnigraph.dev/docs/branching/changes.md).
## Blob commands
```text
omnigraph blob get|stat|put|clear --target NAME|ADDRESS [options]
```
`get` accepts `--branch` or `--snapshot`, `--offset`, `--length`, and
`--out `. `stat` accepts `--branch` or `--snapshot` and `--json`. `put`
reads `--file ` or stdin; `put` and `clear` accept `--branch`,
`--if-match `, `--json` and `--as` (on a served target, an assertion). See [Blob values](https://www.omnigraph.dev/docs/schema/blobs.md).
## Operator configuration
[Operator configuration](https://www.omnigraph.dev/docs/cli/targets.md) owns the version-1 grammar, defaults,
provider protocols, transport limits, credential custody and upgrade procedure.
Use `--operator-config FILE` to select one file; `--config DIR` selects source.
There is no backward compatibility or `target import`. No secrets belong in YAML.
## Managed cluster commands
Account operations use a profile. Resource operations use a managed target
referencing it; see [registration](https://www.omnigraph.dev/docs/cli/targets.md#login-and-registration).
```bash
omnigraph login --create-profile work
omnigraph cluster list --profile work --json
omnigraph target add production --profile work --cluster CLUSTER_ID
```
The target pins cluster ID/incarnation and an optional service-attested endpoint.
Data operations require that endpoint. Refresh is explicit; token issuance cannot
retarget an invocation.
| Managed command | Target and behavior |
| ------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------- |
| `login` / `logout` | Selected `omnigraph` profile (also via a managed target) |
| `cluster create NAME`, `cluster list` | Profile; create returns identity without binding a folder |
| `cluster members list` | Cluster target; list active organization members and identity eligibility |
| `cluster pull --out NEW_DIR [--rev REVISION]` | Download configuration into a new directory |
| `cluster push --config DIR --expected-revision REVISION --message TEXT` | Upload source using an exact revision |
| `cluster plan [--rev REVISION]` | Preview pushed source |
| `cluster apply --plan PREVIEW_ID` | Submit that exact preview |
| `cluster status [DEPLOYMENT_ID] [--wait]` | Observe the cluster or one exact delivery |
| `cluster history`, `cluster cancel DEPLOYMENT_ID` | Observe or cancel managed deliveries |
| `cluster delete --incarnation ID`, `cluster undo-delete --incarnation ID --deletion-id ID` | Assert the target incarnation before lifecycle submission |
| `cluster operation OPERATION_ID [--wait]` | Observe a lifecycle operation using its matching profile or managed target |
Pass `--profile NAME` for account operations and `--target NAME` for resource
operations, or set the corresponding independent default.
`--config DIR` names source files only: managed push accepts it, while managed
plan/apply read pushed source and refuse it. Managed operations never fall back
to direct storage. Self-hosted `validate`, `observe`, `force-unlock`, and
`upgrade-ledger` are unavailable for managed targets.
Authentication uses only the selected owner-local provider. It never falls back
to another profile, source or interactive login. Control-session renewal preserves
the principal and organization and never replays a submitted graph mutation.
Token and helper sources do not establish qualified workload federation.
Managed plan/apply use `--idempotency-key KEY`, `--no-wait`, and
`--timeout SECONDS`. Keep the key and exact request after an uncertain outcome;
replay retrieves the same reservation and changed input refuses. Once an ID is
known, observe it. Polling uses a two-second interval and a default 300-second
local deadline (allowed range 1–3600); a timeout stops the local wait, not the
remote operation. `--no-wait` confirms a reservation, not native acceptance or
activation. Requests refuse redirects and bound their responses.
| Managed deployment result | Exit code |
| -------------------------------------------------------------------------------------------- | --------- |
| Prepared preview, valid observation, confirmed cancellation, or explicit no-wait reservation | 0 |
| Waited result with fresh convergence, active runtime and archived evidence | 0 |
| Native failure, nonconverged result, transport or protocol error | 1 |
| Explicit refusal | 2 |
| Local deadline or unresolved preview replay | 5 |
See [managed lifecycle](https://www.omnigraph.dev/docs/cli/managed-lifecycle.md), [team administration](https://www.omnigraph.dev/docs/cli/managed-team.md)
and [managed data access](https://www.omnigraph.dev/docs/cli/managed-data.md) for custody, permission and bounded
request details. JSON emits one response on stdout; the resolved target,
progress and idempotency information go to stderr.
## Managed data access
A managed target uses the common served graph client, including schema reads,
access introspection, stored-query catalogs, the stdio MCP bridge, branches,
snapshots, history, exports, Blob operations and change feeds. Select a graph
explicitly or use its configured
`default_graph`; discovery only lists available IDs. A separate identity
credential authenticates the data endpoint, and applied Cedar policy decides
permissions. Direct-only maintenance, schema changes and root administration
remain unavailable through a managed data endpoint.
## Confirmation rules
`cleanup` changes nothing until `--confirm` is present. Destructive operations
against non-local storage also require interactive confirmation or `--yes`; in
non-interactive and JSON modes they fail closed. The same non-local consent
rule applies to overwrite loads and branch deletion, verb or statement.
# Open Studio
Source: https://www.omnigraph.dev/docs/cli/studio
`omnigraph studio` opens [OmniGraph Studio](https://github.com/ModernRelay/omnigraph-studio)
on a configured server target, prints the link it opened, and exits:
```bash
omnigraph studio --target staging
omnigraph studio --target staging --graph knowledge --json
```
The command reads the operator configuration and nothing else: it sends no
request to the server, reads no credential and starts no process that serves
a graph. Studio connects to the server itself and obtains the target's
credential through the CLI when the target has one.
## Targets
| Target | Result |
| --------------------------------------------- | -------------------------------------- |
| Server with a token or helper `auth` provider | Opens Studio on that server |
| Server with `auth: {provider: anonymous}` | Opens Studio on that server |
| Direct graph or root, or an inline path | Refuses with `target_kind_unsupported` |
| Managed | Refuses with `target_kind_unsupported` |
Select the target with `--target NAME` or `defaults.target`. `--graph ID` asks
Studio to open that graph; without it Studio opens the target's
`default_graph`, or lets you choose one when the server lists its graphs.
`--profile` and `--as` are refused: Studio acts as the actor its credential
resolves to.
## The link
```text
omnigraph-studio://open?target=NAME&binding=sha256:HEX&home=sha256:HEX[&graph=ID]
```
| Parameter | Meaning |
| --------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `target` | The registered target name |
| `binding` | A digest of what the target resolves to: its kind, endpoint, `auth` provider with its source (the store, an environment variable's name, a file's path or a helper's command), and `transport`. A token's value never enters it, so a rotated token keeps the binding; `default_graph` does not enter it either |
| `home` | A digest of the operator file the command read, at its canonical location: `--operator-config FILE`, else `config.yaml` under `OMNIGRAPH_HOME` or `~/.omnigraph` |
| `graph` | The graph `--graph` named, only when given |
Equivalent spellings of one endpoint or path give the same digest. A link
carries no secret, never names a path or an executable, and opening one runs
no write.
Studio resolves the target itself, through the CLI in its own operator home,
and compares. `omnigraph target inspect NAME --json` reports the same `home`
digest and, for a server target, the same `binding` digest. Studio refuses a
link made in another operator home, for example under a one-off
`OMNIGRAPH_HOME`, and a link whose binding no longer matches because the
target changed since the link was made.
With `--json` the command prints the link and the facts it was built from:
```json
{"protocol":1,"target":"staging","binding":"sha256:…","home":"sha256:…","link":"omnigraph-studio://open?target=staging&binding=sha256:…&home=sha256:…"}
```
`protocol` is `protocols.studio` in [`omnigraph version --json`](https://www.omnigraph.dev/docs/getting-started/install.md#verify).
It versions the link, this report, and the digests `target inspect` reports.
Without `--json` the link is the only line on stdout.
## Platforms
| Platform | How the link opens |
| ----------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
| macOS | `open -b com.modernrelay.omnigraph-studio LINK`: only the application with Studio's bundle identifier |
| Linux and other Unix desktops | The program of the desktop entry `xdg-mime query default x-scheme-handler/omnigraph-studio` names, run directly with the link in place of `%u` |
| Windows | The `omnigraph-studio` URL protocol, only when the registry holds one with an open command |
On Linux the CLI finds the named desktop entry in `$XDG_DATA_HOME/applications`
(default `~/.local/share/applications`), then in each `$XDG_DATA_DIRS/applications`
(default `/usr/local/share:/usr/share`); a `-` in its name may stand for a
subdirectory. It runs the entry's `Exec` command itself, without a shell,
replacing `%u` or `%U` with the link and dropping the other field codes. An
entry that is hidden, runs in a terminal, takes no URL through `%u` or `%U`, or
uses a form the CLI cannot run exactly as written is refused.
The CLI never hands the link to a generic opener such as `xdg-open`, which
passes a link it cannot place to a web browser, for example when no display is
set. When nothing opens the link, because Studio is not installed or no
handler is registered for its scheme, the command still prints the link, says
where to download Studio, and refuses with `studio_unavailable`. With `--json`
that refusal carries `link` and `install`. Studio registers its scheme when
installed on macOS; on Linux and Windows the command opens a link once Studio
registers a handler there.
# Operator configuration
Source: https://www.omnigraph.dev/docs/cli/targets
Use profiles for service/organization access and targets for resource access. This is [target and credential protocol 2](https://www.omnigraph.dev/docs/getting-started/install.md#verify), with YAML `version: 1`.
The operator file is `~/.omnigraph/config.yaml`, or `config.yaml` under
`OMNIGRAPH_HOME`. `--operator-config FILE` replaces it for one invocation;
files are never merged or discovered by searching parent directories.
`--config DIR` remains deployment source, not operator configuration.
Operator YAML and its parent may be read-only; authentication keeps mutable state in
writable `OMNIGRAPH_HOME` (normally `~/.omnigraph`). Explicit configuration edits
require writable YAML. Moving the file does not automatically transfer saved credentials.
## Configure destinations
```yaml
version: 1
profiles:
work:
api: https://platform.omnigraph.dev
organization_id: org_example
auth: {provider: omnigraph}
targets:
production:
kind: managed
profile: work
cluster_id: cluster_example
cluster_incarnation: incarnation_example
endpoint: https://cluster.example.com
default_graph: knowledge
standalone:
kind: server
endpoint: https://graph.example.com
auth: {provider: token, source: {store: {}}}
scratch:
kind: direct
scope: graph
uri: file:///srv/graphs/scratch.omni
defaults:
output: table
```
A profile requires `api`, `organization_id` and one `auth` provider object. Each profile represents one
identity; use separate profiles for different people or automation identities, even within the same
organization. A managed target references that profile and pins `cluster_id` and `cluster_incarnation`. Its
optional `endpoint` is required for data operations; without it the registration is control-only. Server
targets own `endpoint` and one `auth` provider directly. Use separate server targets
for different identities, even when their endpoints match. Direct targets have `scope: graph` or `scope: root` and an
absolute local `file://` URI without remote authority, query or fragment. A root may declare `default_graph`;
a direct graph may not.
Names match `[A-Za-z][A-Za-z0-9_.-]*` and are case-sensitive. The
[JSON Schema](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/cli/operator-config.schema.json) describes the fields;
`omnigraph config validate` also checks references and cross-field constraints.
The whole document must be valid, including unselected entries. Unknown fields,
versions, duplicate keys, YAML tags, anchors, aliases, merge keys and implicit null defaults refuse.
Organization IDs contain 1–128 ASCII letters, digits, underscores or hyphens;
cluster IDs and incarnations use the same characters with a 256-character limit.
Graph IDs use the server's rule: 1–64 ASCII letters, digits or hyphens, excluding
`policies`, `healthz`, `openapi` and `graphs`. Served targets additionally accept
the system scope `og_catalog`; direct-root targets do not. Validation checks these
rules for defaults and alias graph selectors before any network or credential access.
There is no interpolation, include or inheritance. The input limit is 1 MiB.
Relative token and CA paths resolve beside the selected operator file.
Credential, helper and CA availability are checked only for selected operations.
## Selection and defaults
| Selection | Precedence and boundary |
| ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| Account operations | `--profile NAME`, then `defaults.profile` |
| Resource operations | `--target NAME`, then `defaults.target` |
| Managed authentication | The selected profile's single `auth` provider |
| Standalone authentication | The selected server target's single `auth` provider |
| Graph | `--graph ID`, then target `default_graph`; otherwise graph-scoped operations refuse |
| Read output | Explicit `--format`, then `defaults.output`, then `table`; `--json` selects JSON where offered. `toon` encodes the same read document as `json` |
Profile and target selectors are mutually exclusive; their defaults never infer each other. Managed targets
always use their pinned `profile`; neither `defaults.profile` nor a global `--profile` can override that
binding. Every selected owner has exactly one authentication provider. There is no last-used identity or
implicit single-profile selection. A direct graph rejects a separate graph selector. Discovery never selects
the first graph. Graph IDs are exact API identifiers, not display-name lookups; cluster incarnation pinning
does not protect against recreation of a graph with the same ID.
Inline targets are local paths or `file://` URIs, such as `--target ./graph.omni`,
and an [S3 cluster root](https://www.omnigraph.dev/docs/operations/s3-cluster-roots.md) for deployment,
recovery and maintenance commands. Bare names select registered targets. Remote endpoints
must be registered. `init ./graph.omni --schema schema.pg` keeps its local shorthand.
Defaults and aliases require registered names. Registered destinations have no environment overrides.
`defaults.output` accepts only `table`, `json`, `jsonl`, `csv`, `kv`, or `toon` and
applies to supported read-result formatting. Action receipts retain their own
contracts. Unsupported formats refuse. Diagnostics go to stderr, results to stdout.
## Authentication providers
Each profile or server target chooses one provider:
| Access method | `auth` value |
| -------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Managed device login | `{provider: omnigraph}` |
| Stored token | `{provider: token, source: {store: {}}}` |
| Environment token | `{provider: token, source: {env: OG_SERVICE_TOKEN}}` |
| Mounted token | `{provider: token, source: {file: /var/run/omnigraph/token}}` |
| Credential helper | `{provider: exec, command: [/usr/local/bin/credential-helper, get], timeout: 30s}` |
| Managed OAuth client credentials | `{provider: oauth_client_credentials, token_endpoint: https://auth.example.com/oauth2/token, client_id: client_example, client_secret: {env: OG_CLIENT_SECRET}}` |
| Explicit anonymous server | `{provider: anonymous}` |
`omnigraph` and `oauth_client_credentials` are managed-only; `anonymous` is
standalone-only. Names grant no privileges, and token/exec sources do not qualify
arbitrary workload federation.
Token profiles declare exactly one source. `store` uses secure local custody;
`env` reads only the named variable; `file` reads one bounded regular-file
snapshot, allowing symlink-based atomic rotation. Missing or empty sources fail
without fallback. Environment names match `[A-Za-z_][A-Za-z0-9_]*`; there is no
shell expansion. OmniGraph never writes or removes externally owned token files.
Tokens are bounded to 64 KiB and must be valid bearer tokens. Stdin/file input
may remove one terminal LF or CRLF; environment values must contain only the
token. Installation does not verify server acceptance or establish the principal
of an opaque standalone token. A new invocation can observe source rotation.
Exec uses an absolute executable path and argv, never a shell. Timeout is
`1s`–`120s`, default `30s`; stdout and stderr are each bounded to 64 KiB.
The helper receives one version-1 JSON request on stdin with purpose, endpoint,
`interactive: false` and applicable identity constraints. It returns one object
with `version: 1`, `credential: {type: bearer, token: ...}` and optional RFC 3339
`expires_at`. Unknown fields, expired results and unsupported types refuse.
Expiry is rechecked before authenticated dispatch, including after discovery.
Renewal can precede submission but never replay a mutation. Managed control and
data credentials have independent expiry times.
Without expiry the result is not cached across invocations. Timeout terminates
the helper process group; stderr is not echoed in routine errors.
Output cannot change routing or prove identity. Helpers inherit the process
environment and are trusted programs, not sandboxed plugins. Inspection and
validation never execute them.
The exec adapter currently requires Unix process-group support; other platforms
refuse it rather than launch a helper without bounded cancellation.
Never put tokens, refresh tokens or private keys in YAML. Persistent login fails
if secure custody is unavailable; there is no plaintext fallback. Resource
commands never start interactive login. Managed access exchanges the selected
profile credential for a cluster-bound data credential; control and refresh
tokens never go to the graph endpoint. Service and graph policy still authorize
operations independently of credential acquisition.
For managed resource commands and managed-target credential export, env/file/exec/OAuth
profiles keep exchanged credentials in invocation memory. Human profiles, stored
tokens and existing providers' explicit `cluster token` require secure storage.
OAuth always uses invocation memory ([workload setup](https://www.omnigraph.dev/docs/cli/managed-team.md#configure-an-unattended-workload-profile)); store failure never changes custody.
## Login and registration
```bash
omnigraph login --create-profile work
omnigraph login --api https://control.example.com --create-profile private
omnigraph login --profile work
omnigraph cluster list --profile work --json
omnigraph target add production --profile work --cluster CLUSTER_ID
omnigraph target inspect production --resolved --json
omnigraph target refresh production
```
`login --create-profile NAME` authenticates and verifies the organization before
creating a profile. Existing names refuse; routing defaults stay unchanged.
Only creation accepts `--api URL` (default `https://platform.omnigraph.dev`)
and `--organization ID`.
`login --profile NAME` signs in to an existing profile. Bare `login` uses
`defaults.profile`; without a selection it explains how to select or create one.
Neither form implicitly creates a profile or selects the only configured entry.
Login/logout require `provider: omnigraph`, directly or via a managed target's pinned
profile. Static tokens use `credentials`.
Target registration uses authenticated discovery to save exact cluster identity
and its service-attested endpoint, if available. It creates no remote cluster.
Refresh verifies the same incarnation before changing the route pin; a missing
endpoint preserves the old pin and reports `endpoint_available: false`.
Normal requests never adopt changed endpoints or follow redirects.
`target list` shows each destination, its managed profile, and the default marker.
`target list`, `target inspect NAME [--resolved]` and `config validate` are offline, with optional
`--json`. `target inspect --json` also reports `home` and, for a server target, `binding`: the digests
a [Studio link](https://www.omnigraph.dev/docs/cli/studio.md#the-link) carries. `target add NAME` without `--cluster` reads a complete
target JSON object from piped stdin (64 KiB limit); terminal or empty input produces setup guidance.
Managed add uses `--profile` or `defaults.profile`. `target remove NAME` preserves credentials and
journals; references must remain valid. Add/remove never switch defaults.
Discovery-backed add and refresh contact the service. Resolved inspection expands
the profile and provider; identity is unknown and transport availability is not checked.
Logout clears local session and derived credentials and reports whether remote
revocation was confirmed; independently valid data tokens may remain usable.
Preserve unresolved operation evidence across logout and configuration changes.
To deliberately replace a profile's principal, confirm the **new** principal ID:
```bash
omnigraph login --profile work --replace-principal PRINCIPAL_ID
```
Replacement requires an existing profile or managed target and a newly verified
principal matching the supplied ID. It refuses `--create-profile`, `--api` and
`--organization`, invalidates the credential generation and clears derived data
credentials without rerouting existing operations.
## What you may do at a target
```bash
omnigraph access --target standalone --graph knowledge --branch main --json
```
`access` reports your actor and permissions for direct, server and managed graphs.
Use `--branch` for branch permissions and `--target-branch` for create/merge
permissions. JSON uses `protocol: 2` and `actor_id`; see
[access semantics](https://www.omnigraph.dev/docs/operations/policy.md#learn-your-access) and [JSON protocols](https://www.omnigraph.dev/docs/getting-started/install.md#verify).
## Rename registrations and profiles
```bash
omnigraph config rename-profile work company
omnigraph config rename-target production live
# After an interrupted rename, select the same operator file:
omnigraph config recover-rename --operator-config /path/to/config.yaml
```
These local commands preserve credential binding IDs. Profile renames update managed references and the
profile default; target renames update aliases and the target default.
Destination-name or retained-custody collisions refuse. Use these commands instead
of manually renaming YAML keys, which does not move custody.
An interrupted rename blocks normal config use until `config recover-rename` finishes it. Conflicting
subsequent edits refuse recovery and preserve the journal for review. Renames and rename recovery have human
output only, with no `--json`.
## Manage credentials
```bash
printf '%s\n' "$SERVER_TOKEN" | omnigraph credentials set --target standalone --stdin
omnigraph credentials status --target standalone --json
omnigraph credentials remove --target standalone
# Pipe export directly to the intended consumer, never to a file.
omnigraph credentials export --target standalone | consumer
```
Credentials require one explicit owner selector; defaults do not apply.
Managed credentials use `--profile NAME`:
```bash
omnigraph credentials status --profile work --json
printf '%s\n' "$CONTROL_TOKEN" | omnigraph credentials set --profile automation --stdin
```
Standalone credentials use `--target NAME`; managed-target export
uses its pinned profile. Managed set/status/remove require `--profile`.
Direct targets have no HTTP credentials. Set/remove require `token` with
`source: {store: {}}`; set requires `--stdin`, plus `--replace` for existing entries.
Neither edits YAML. Remove is local and idempotent, not remote revocation.
Status performs no network request, refresh, login or helper execution. It reports source availability and
locally known metadata, never values. Secure-store availability and identity are reported as unknown without
prompting or reading secrets. File status does not read contents. Availability is not validity.
Export currently requires a Unix FIFO pipe, refusing other platforms, terminals and regular files before
acquisition. Success writes only a bearer access token and newline, ignoring
output defaults. It may refresh or invoke the selected helper but never starts
interactive login. `--profile` exports control access; a managed `--target`
exports only exchanged cluster data access. Anonymous and refresh credentials
cannot be exported. The receiving process can copy or disclose the token.
## Transport and capability boundaries
Remote endpoints require HTTPS. The only HTTP exception is an explicitly
anonymous loopback standalone server. Optional `transport` belongs independently
to a profile's API or target's data endpoint; it is not inherited.
API/endpoint URLs may include a base-path prefix, which stays inside the
credential scope. Userinfo, queries, fragments, traversal and ambiguous encoded
separators refuse. TLS verifies certificates and hostnames with system roots by default.
`tls.ca: {mode: append, file: company-ca.pem}` adds a PEM CA bundle;
`mode: replace` replaces destination roots. There is no insecure mode.
Proxy defaults to `{mode: direct}`, ignoring ambient proxy variables.
`{mode: url, url: http://proxy.example.com:8080}` selects a proxy explicitly;
`{mode: environment}` opts into captured variables. HTTPS uses
`https_proxy`/`HTTPS_PROXY`; loopback HTTP uses lowercase `http_proxy` only.
Conflicting spellings refuse. `no_proxy`/`NO_PROXY` supports `*`, exact hosts/IPs
and leading-dot DNS suffixes with optional ports; CIDR is unsupported.
Empty proxy values mean direct; bracket IPv6 literals in bypass entries.
`ALL_PROXY` is unsupported. Proxy URLs accept only HTTP/HTTPS and forbid credentials, non-root paths,
queries and fragments. Version 1 rejects custom destination CA plus HTTPS proxy.
HTTP CONNECT proxies support custom destination CAs. Identity-provider
traffic does not inherit API transport settings.
Standalone requests stay within the selected endpoint prefix. Requests cannot
borrow credentials from, or carry the parent's token into, a more-specific
registered endpoint. Targets at the same root do not override the selected
owner. These boundaries are captured for the invocation.
Version 1 does not support other cloud-direct targets, catalogue targets, Unix sockets,
mTLS, tunnels or arbitrary private identity-provider transport. Server storage
and deployment retain separate contracts. This rewrite does not qualify BYOC
private routes or workload federation. Managed and standalone targets share
served graph operations subject to capabilities and policy; maintenance, schema
changes and direct-root administration keep their storage restrictions. Remote
failures never trigger direct storage fallback.
The old standalone `--target og_catalog` shorthand is not a built-in target;
catalogue reads through a registered server still use `--graph og_catalog`.
## Read aliases
An alias has `target`, `query`, optional `params`, and required explicit `graph`
for multi-graph targets. A direct-graph alias must omit `graph`, but execution
still refuses because the stored-query registry is served-only. Aliases never
inherit `default_graph`, contain source code or mutate data. Caller target/graph
overrides refuse. Aliases use their target's authentication; managed aliases keep
the target's pinned profile. `--params`
and `--params-file` must contain a JSON object. Its fields shallowly replace
defaults as typed values, never source interpolation. Arrays, scalars and null
are refused. Alias `args` and `format` fields and positional arguments are unsupported.
## Upgrade from the old format
For released **0.13 → 0.14**, follow the [upgrade checklist](https://www.omnigraph.dev/docs/operations/upgrade-0.14.md)
with paired YAML and commands. Old resource-binding `profiles` become `targets`;
new profiles hold managed sign-in settings. There is no config importer.
For intermediate 0.14 formats, replace `connections`/account targets with profiles,
target `connection` with `profile`, and `defaults.connection` with `defaults.profile`.
Flatten each `auth` to one provider and remove `default_auth`. Split identities into
separate profiles or server targets; reinstall stored server tokens with
`credentials set --target NAME --stdin`. Use `login --create-profile NAME` for setup.
Old `--connection`, `--auth`, `config rename-auth`, `target register` and
`login --register` refuse. Remove alias `args`/`format`; use explicit `--as` for direct actors.
Preserve old homes and journals; config replacement changes no graph data. Finish
interrupted old server renames with the previous CLI's `config recover-rename`.
The new CLI refuses incompatible journals; compatible profile renames can recover.
For version-2 folder contexts carrying an organization ID, recover lifecycle evidence:
```bash
omnigraph config recover-operations --source /path/to/old-workspace --profile work --json
```
It verifies API/organization against `.omnigraph` evidence, durably copies matching
pending/accepted records, then retires originals. Conflicts refuse; preserve both
copies. It neither converts config nor submits operations. Resume with the original
verified principal/key; observe accepted IDs. Exclude older CLIs after migration.
The version-1 context shipped in 0.13 lacks an organization ID and is unsupported.
Settle its pending/accepted work with the originating build before cutover; preserve
identity, keys and journals. Never edit versions or invent identity fields to import.
# Cluster configuration reference
Source: https://www.omnigraph.dev/docs/clusters/config
`--config DIR` reads desired configuration from a directory containing
`cluster.yaml`. `--target NAME|ADDRESS` separately selects the backend for
plan, apply and observation. Validation needs only source files:
```bash
omnigraph cluster validate --config ./company-brain
```
`--config` defaults to the current directory. Unknown fields and duplicate YAML
keys are errors so that misspelled intent is never ignored.
## Complete shape
```yaml
version: 1
metadata:
name: company-brain
# Optional assertion: must match the selected target for plan/apply.
storage: s3://company-data/omnigraph/company-brain
state:
backend: cluster
lock: true
providers:
embedding:
default:
kind: openai-compatible
base_url: https://api.example.com/v1
model: text-embedding-3-large
api_key: ${EMBEDDING_API_KEY}
graphs:
knowledge:
schema: knowledge.pg
queries: queries/
embedding_provider: default
external_blobs:
allow:
- base: s3://company-assets/knowledge/
scope: server_safe
policies:
graph-access:
file: graph.policy.yaml
applies_to: [knowledge]
server-access:
file: server.policy.yaml
applies_to: [cluster]
```
## Top-level fields
| Field | Required | Meaning |
| --------------------- | -------: | --------------------------------------------------------------------------------------- |
| `version` | yes | Configuration schema; currently `1` |
| `metadata.name` | no | Display name |
| `storage` | no | Optional root assertion; must match the selected target for direct or server plan/apply |
| `state.backend` | no | Omit or set to `cluster` |
| `state.lock` | no | Exclusive writer admission; omit or set to `true`; `false` refuses execution |
| `providers.embedding` | no | Named embedding provider profiles |
| `graphs` | no | Graph declarations keyed by graph ID |
| `policies` | no | Policy bundles keyed by bundle name |
Credentials are process configuration and must not appear in
`cluster.yaml`.
## Graphs
Each graph requires a schema file:
```yaml
graphs:
knowledge:
schema: knowledge.pg
```
Optional fields:
| Field | Meaning |
| -------------------- | ---------------------------------------------------------- |
| `queries` | Stored-query files, directories, or explicit name mappings |
| `embedding_provider` | Name under `providers.embedding` |
| `external_blobs` | Allow-list for new external Blob references |
Query declarations support three forms:
```yaml
# Every declaration in top-level *.gq files in a directory
queries: queries/
# Every declaration in these files or directories
queries: [people.gq, reports/]
# Explicit registry names
queries:
find_experts:
file: knowledge.gq
```
Unreadable files, parse errors, duplicate query names, and queries that do not
type-check against the graph's desired schema fail validation.
## Embedding providers
Provider `kind` may be `openai-compatible`, `openai`, `gemini`, or `mock`.
Real providers require `api_key: ${ENVIRONMENT_VARIABLE}`; inline secrets are
rejected. The serving process resolves the environment variable at boot and
during live deployment preflight for affected graphs. `cluster validate`,
`plan`, and direct apply do not resolve it. Vector dimensions remain part of
the graph schema.
See [Embeddings](https://www.omnigraph.dev/docs/queries-search/embeddings.md) for provider behavior.
## External Blob references
New external references are denied unless their normalized URI falls under an
allowed base:
```yaml
external_blobs:
allow:
- base: s3://company-assets/knowledge/
scope: server_safe
```
`server_safe` permits the base for a served graph. `embedded_only` is for an
embedded host and may permit a local `file://` directory; it is not installed
by the HTTP server or direct-store CLI. Bases must be absolute, non-overlapping,
and free of credentials, query strings, fragments, and path traversal.
A base must also name storage outside the selected cluster root. That root
holds every graph and the applied state, and ingress reads with the process's
own storage credentials, so a base over it would let any writer copy another
graph's data or the cluster ledger into a readable Blob value. Plan and apply
refuse such a base with `external_blob_base_overlaps_storage_root`, whatever
its scope. Offline validation checks overlap when `storage` declares a root;
otherwise the destination check waits for plan/apply. Put external objects under a sibling prefix instead, for
example `s3://company-assets/cluster-external/` beside
`storage: s3://company-assets/cluster`. A server that finds an overlapping
`server_safe` base in the applied state quarantines that graph and serves the
others; if no applied graph is left to serve, startup fails with
`cluster_no_healthy_graphs`. An embedded handle refuses a policy whose base
overlaps its own graph root.
A base is compared only with a storage root of its own kind: an `s3://` base
with an `s3://` root, a `file://` base with a local root. When the root is
spelled with a path component a base URI cannot express (an empty component
such as `s3://bucket/a//cluster`, or a percent sign in a local path), a
same-kind base is refused with `external_blob_storage_root_uncomparable`,
because disjointness cannot be proven. Moving the base does not clear that
code; the storage root spelling does.
The allow-list controls which external objects an authorized writer may cause
the process to inspect: a new external Blob reference, and the source of a
`load csv` statement, which the engine reads in full once
([CSV loading](https://www.omnigraph.dev/docs/queries-search/mutations.md#csv-loading)). Cedar policy separately
decides who may write. See [Blob values](https://www.omnigraph.dev/docs/schema/blobs.md).
## Policies
```yaml
policies:
graph-access:
file: graph.policy.yaml
applies_to: [knowledge, catalog]
registry-access:
file: server.policy.yaml
applies_to: [cluster]
```
A bundle targets either graph IDs or the `cluster` server scope, never both.
Only one bundle may bind a given graph or the cluster scope. See
[Authorization](https://www.omnigraph.dev/docs/operations/policy.md).
## Storage
Plan and apply always use the selected direct-root or server target. Omitting
`storage` makes the bundle portable between explicitly selected destinations.
If present, `storage` is an equality assertion: a mismatch refuses before
effects. Remote operations require an absolute path or `file://`, `s3://` or
`az://` URI and read only source files from the caller's filesystem.
Registered direct targets accept local files only. The storage assertion above
describes the deployment root; it does not select a target. An inline `s3://`
root reaches only the commands listed in
[S3 cluster roots](https://www.omnigraph.dev/docs/operations/s3-cluster-roots.md).
Cloud storage credentials and provider settings belong to the server or embedded
initializer. Azure remains a qualification preview and requires its admission
wrapper for every writer; see [Deployment](https://www.omnigraph.dev/docs/getting-started/deployment.md#azure-blob-preview).
## Declared paths
Every `schema`, `queries`, and policy `file` path is resolved against the
directory that holds `cluster.yaml`. A relative path must stay inside that
directory: a `..` segment is refused with `config_path_escape`, and a path
that reaches its file through a symbolic link, on the way to it or as a
query file discovered inside a declared directory, is refused with
`config_path_symlink`. Each diagnostic names the setting that declared the
path. The bundle is one directory of files read exactly as declared, so what
gets applied never depends on something outside it.
An absolute path is accepted as given and is not checked for either shape.
Prefer relative paths; they are what keep a bundle portable and hermetic.
## Command behavior
| Command | Changes graph or cluster state? | Use |
| ---------------- | ------------------------------: | -------------------------------------------------------------------------------- |
| `validate` | no | Parse and type-check the declaration |
| `plan` | no | Preview desired differences; unsupported deployment effects still refuse |
| `apply` | yes | Converge graph inventory, schemas, queries, policies and provider/Blob settings |
| `status` | no | Read recorded deployment and lock status; server status also observes activation |
| `observe` | no | Report current observations without changing authority |
| `force-unlock` | yes | Remove one proven-stale lock by exact ID |
| `upgrade-ledger` | state only | Convert a stopped legacy ledger without resetting graph data |
All execution uses ledger v2 and requires `state.lock: true`. A server target
submits apply to its existing writer and activates without restart. Direct
apply owns exclusive admission and requires an explicit handoff before serving.
Retained graph roots and storage formats stay fixed. Policy/provider/Blob bindings
can change; removing a graph declaration deletes its managed storage and history. See
[deployment boundaries](https://www.omnigraph.dev/docs/clusters.md#deployment-boundaries).
Apply does not load rows or start servers. Root-addressed status, reconciliation
and conversion do not use `cluster.yaml`; see [the deployment workflow](https://www.omnigraph.dev/docs/clusters.md).
## Limits
Configuration loading refuses limits before schema effects. Shared files with
identical bytes count once toward the aggregate source limit.
| Resource | Limit |
| --------------------------------------------------------------------- | ----------------------------------: |
| Each `cluster.yaml`, schema, query or policy source | 1 MiB |
| Distinct source bytes in one captured bundle | 8 MiB |
| Declared resources (each graph and schema count separately) | 4096 |
| Query discovery paths plus directory entries, including ignored files | 4096 |
| Encoded immutable deployment bundle | 16 MiB |
| Encoded cluster ledger | 16 MiB |
| Lock metadata read | 64 KiB |
| Outstanding deployments | 1 |
| Retained deployment results | 32 records, 4 MiB total, 1 MiB each |
| Deployment actor / resource address / canonical root | 256 / 512 / 4096 UTF-8 bytes |
Deployment admission checks the actual encoded prepared intents and reserves
ledger/result space to record completion before accepting schema effects.
JSON escaping and prepared state can make a deployment exceed its encoded limit
even when raw source bytes fit. Result history may evict older completed records
to admit a new deployment; outstanding authority is retained. These bounds do
not cap graph data, native manifest-history scans or total process memory.
See [Operating a cluster](https://www.omnigraph.dev/docs/clusters.md) for the end-to-end workflow.
## Operator targets
The operator file declares destinations separately from this source bundle.
See [Operator targets](https://www.omnigraph.dev/docs/cli/targets.md) for the format, provider settings,
credentials, administration and migration.
# Operating a cluster
Source: https://www.omnigraph.dev/docs/clusters
An OmniGraph cluster is a declarative bundle of graphs, schemas, stored queries
and authorization policies. Apply converges graph inventory, schemas, queries,
policies and provider/Blob settings. Submit to the running server to activate
changes without a restart.
Use a cluster for a multi-graph server or shared operational configuration. For
one local graph, the [quickstart](https://www.omnigraph.dev/docs/getting-started/quickstart.md) is simpler.
## Create a bundle
```text
company-brain/
├── cluster.yaml
├── knowledge.pg
├── queries/
│ └── people.gq
├── cluster.policy.yaml
└── graph.policy.yaml
```
```yaml
version: 1
metadata:
name: company-brain
graphs:
knowledge:
schema: knowledge.pg
queries: queries/
policies:
cluster-access:
file: cluster.policy.yaml
applies_to: [cluster]
graph-access:
file: graph.policy.yaml
applies_to: [knowledge]
```
Paths are relative to `cluster.yaml`. Its configuration version is independent
of the deployment ledger version. A declared `storage:` is an equality assertion:
it must match the selected root, and never redirects the command. For direct
operations, relative storage paths resolve against the source directory;
server submission requires an absolute assertion or no `storage:`. Omitting
`--config` on source-bearing direct/server plan or apply uses the current
directory (`.`). `cluster validate` reads source only and needs no target. The [configuration reference](https://www.omnigraph.dev/docs/clusters/config.md)
covers storage roots, embedding providers, external Blob policy and limits.
For server-owned deployment, the applied policy must grant the operator
`config_manage` at cluster scope. Schema changes and graph deletions additionally
require `read` and `schema_apply` on the affected graphs. Direct deletion also
enforces an installed graph policy and requires an authorized `--as` actor.
Deployment status requires `config_manage`, not unrelated data access. New graphs
need suitable declared policies too.
The server derives the actor from its bearer token; there `--as` only asserts it.
## Bootstrap a cluster
```bash
mkdir -p ./company-data
omnigraph cluster validate --config ./company-brain
omnigraph cluster plan --target ./company-data --config ./company-brain
omnigraph cluster apply --target ./company-data --config ./company-brain --as act-alice --json
```
`--config` names the source directory containing `cluster.yaml`; `--target`
selects the destination independently. Here `./company-data` is an inline direct
root target. A configured target name or `defaults.target` works too. No folder
file or `storage:` declaration selects a target. Every backend command reports
its resolved target on stderr before execution.
Fresh apply creates the ledger and declared graphs in the selected root, captures
source bytes before execution and prints the original `Deployment-ID` with exact
outcomes. Use `load` or `mutate` for rows.
Direct apply retains its admission lock after completion. Establish that the
owner and its accepted I/O have settled, then follow
[ownership transfer](https://www.omnigraph.dev/docs/getting-started/deployment.md#writer-topology) using the exact printed
lock ID before starting the server:
```bash
omnigraph --target file:///srv/company-brain cluster force-unlock ''
OMNIGRAPH_SERVER_BEARER_TOKENS_JSON='{"act-alice":"secret"}' \
omnigraph-server --cluster file:///srv/company-brain --bind 0.0.0.0:8080
```
Use the actual root printed by apply. A directory boot resolves its storage root
through `cluster.yaml`; a root URI boots directly from applied resources. Editing
local files alone never changes serving behavior. See
[HTTP server](https://www.omnigraph.dev/docs/operations/server.md) for authentication and routes.
### First boot from a bootstrap receipt
Administrators using the `omnigraph-cluster` library can prepare a fresh S3
cluster with `bootstrap_serving`, containing only its initial cluster management
policy. After that initializer has stopped, supply its exact receipt at first boot:
```bash
OMNIGRAPH_SERVER_BEARER_TOKENS_JSON='{"act-alice":"secret"}' \
omnigraph-server --cluster s3://company-data/company-brain \
--bootstrap-handoff /run/omnigraph/bootstrap-receipt.json
```
The receipt must be a regular strict-JSON file of at most 16 KiB matching the
root and completed empty bootstrap. The server claims ownership once before
listening. Replay, mismatched state and uncertain claims refuse. Failure after
claiming retains ownership; investigate that attempt before any ownership change.
The receipt grants no user permission; configure authentication normally. Protect
both receipt and boot configuration. Only fresh S3 first boot is supported, with
all other writers excluded; it is not a restart, recovery or fencing mechanism.
## Deploy without restarting
Edit and validate the bundle, then submit it to the running owner:
```bash
omnigraph cluster validate --config ./company-brain
omnigraph cluster apply \
--target production --config ./company-brain --json
```
`production` names a server target with its own `auth` provider.
Configure it and install a token or declare an external source using the
[operator reference](https://www.omnigraph.dev/docs/cli/targets.md). No implicit token environment override
or static-token login remains.
With a server target, omitted `storage` binds the bundle to the selected server’s
canonical root. An explicit absolute `storage` must match that root; relative
storage paths refuse. The CLI reads only local source files and needs no local
mount or storage credentials for the server’s root.
Preview the same local configuration through the server before applying:
```bash
omnigraph cluster plan --target production --config . --json
omnigraph cluster apply --target production --config . --timeout 1800 --json
```
The served plan lists resource changes and schema migrations, including the exact
managed root and history that a graph removal deletes. It writes nothing and keeps
serving admission open. Its ledger CAS and input digest identify an observation;
apply rechecks current authority and physical execution eligibility.
The CLI prints the deployment ID before submission. The server keeps its PID,
listener and writer ownership. It closes admission on affected graphs,
finishes their admitted requests, publishes schema changes, and activates
matching schemas, queries and runtime permissions together. Unaffected graphs
keep serving.
Graph additions and removals use the same deployment; see
[deletion semantics](#deployment-boundaries) before removing a declaration.
The response separates the durable deployment result from `active`, which means
that result's affected bindings are installed in this process. An unrelated
blocked graph does not invalidate that activation. Default apply waits for
convergence and activation by polling the original ID. `--no-wait` instead returns
after durable acceptance; drain and preparation may precede that acknowledgment.
`--timeout SECONDS` bounds caller waiting, including acceptance (default 300,
maximum 3600). Expiry exits 5 with the original ID and last observation; it does
not cancel execution or prove failure. After reporting expiry, the CLI does not
wait for an unfinished background DNS lookup before exiting. A failed recovery
hint also preserves the server's original refusal, including an outstanding
deployment ID. Resume observation with:
```bash
omnigraph cluster status --target production --deployment-id ID --wait --timeout 1800 --json
```
The exact response contains `deployment`, `active` and `in_progress`. Default
apply and status `--wait` return at activation, which can precede release of
deployment admission. Repeat exact-ID status reads until `in_progress: false`
before submitting another deployment; this confirms the observed deployment has
released its admission. General cluster status retains its aggregate `status` object. An authenticated submitter
can read its own durable receipt even after its deployment removes its management
permission; general status and later deployments still require current permission.
Lost submission responses are followed only by original-ID reads, never automatic
resubmission. While waiting, observation retries transient HTTP 429/503 responses
and interrupted response bodies within the same budget; malformed receipts fail immediately.
Expired receipts and unknown outcomes require investigation.
Each graph publishes atomically; deployment
across multiple graphs is not one transaction. Query-only changes create no graph
commit and also work with multiple branches. Schema changes remain main-only
and require a single live branch.
Policies, provider definitions and graph bindings, and external-Blob rules can
change on existing graphs. Current permissions authorize the deployment; proposed
permissions cannot authorize themselves. Provider changes do not re-embed stored
vectors. Roots, format and credential/trust configuration stay fixed.
A refusal before effects restores unchanged serving views, including after a
drain timeout. That timeout bounds draining admitted requests; once drained,
the server owns preparation, completion and activation through the existing
shutdown boundary. There is one deployment protocol and no legacy execution fallback.
Upgrade the CLI, server and cluster tools together. Finish outstanding deployments
with the build that accepted them before upgrading: captured inputs are exact
and are not translated into a different request. Use the stopped-ledger upgrade
command to remove obsolete completed-result runtime fields; graph data and exact
achieved receipts are preserved. The graph storage format is unchanged.
## Direct deployments and conversion
With a direct root target, apply executes under its own exclusive admission and requires
the serving owner to have stopped and handed off the lock. Start the server after
settlement to activate the applied revision. Direct apply never takes over a live
server or writes around its lock.
Use explicit stopped-ledger conversion for a v1 ledger or a v2 ledger whose
completed receipts still contain obsolete runtime activation fields:
```bash
omnigraph --target file:///srv/company-brain \
cluster upgrade-ledger --writers-stopped --json
```
Stop serving, writers and maintenance and establish prior graph/control I/O
quiescence first. Conversion preserves rows, graph identities, branches, history
and applied resources. It does not reset graphs, replay old work or convert graph
storage formats. There is no automatic migration or v1 execution fallback.
For v2, conversion removes only those obsolete runtime fields, preserving the
ledger identity, next deployment sequence and exact achieved receipts. Outstanding
deployments and unsupported receipt shapes refuse; finish accepted work with
its originating build before upgrading.
Normal reads report `ledger_upgrade_required` with the stopped-upgrade command
for a recognized prior receipt; malformed or unknown state remains an error.
A completed read-only preflight refusal releases a newly acquired direct lock.
Accepted work, cancellation and uncertain effects retain it for reconciliation.
`--as` labels a storage-owning operator; installed graph policies still govern
schema effects. It is not remote authentication.
## Inspect and recover a deployment
A lost connection does not mean failure. Observe the original ID:
```bash
omnigraph cluster status \
--target production --deployment-id '' --json
```
An output failure also returns nonzero without proving that deployment failed.
Served apply must write and flush its deployment ID before submitting. Direct
apply finishes its one deployment attempt even if its identity diagnostic cannot
be written, then returns the output error with the original ID, root and lock
information. Observe that identity before taking further action; failed output
does not cause automatic resubmission.
Status never reads source files. `cluster apply --deployment-id ID` without
`--config` is also source-free: on a direct root it observes the recorded ID,
and `--writers-stopped` permits reconciliation only with an explicit `--target`.
For a new source-bearing submission with a caller-chosen ID, provide both
`--config DIR` and `--deployment-id ID`. The writer attestation cannot accompany
source input. On a server target, use `cluster status --deployment-id ID` for
observation; an ID-bearing apply requires explicit `--config`.
Status is read-only. `active` is false for an older result after a newer revision
activates. Only the current runtime can prove activation; a durable completion
receipt alone cannot. Repeating apply with the same ID and identical captured input
returns its recorded outcome; it never executes again. Different input refuses.
If the server stopped with an outstanding deployment, inspect the storage root:
```bash
omnigraph --target file:///srv/company-brain \
cluster status --deployment-id '' --json
```
Establish prior-owner and accepted-I/O quiescence, exclude concurrent admissions
and unlocks, then reconcile that exact ID:
```bash
omnigraph --target file:///srv/company-brain cluster force-unlock ''
omnigraph --target file:///srv/company-brain \
cluster apply --deployment-id '' --writers-stopped --json
```
An S3 root works the same way inline, for example
`--target s3://bucket/company-brain`; see
[S3 cluster roots](https://www.omnigraph.dev/docs/operations/s3-cluster-roots.md).
Reconciliation uses captured input and exact publication evidence; it never
replays an uncertain schema or graph-creation invocation. Unstarted schema or
graph-creation work can be recorded as not attempted. A partial graph birth can
be abandoned only when its exact unpublished, empty artifacts belong to that
attempt. A foreign or committed graph is never reset. Unknown outcomes stay
outstanding and block new writes. A later creation may reuse an empty local
directory tree left by that cleanup; files, symlinks and cloud markers still refuse.
A settled partial result permits a corrective successor from achieved state;
it does not roll back graphs that committed. Recovery retains a new admission
lock, so perform ownership transfer before starting another owner. Never
allocate a new ID to retry an unknown outcome. Result eviction reports acceptance
and outcome as unknown; it never authorizes replay. See [limits](https://www.omnigraph.dev/docs/clusters/config.md#limits).
## Deployment boundaries
`cluster apply` creates graphs, deletes removed graphs, and updates schemas,
stored queries, policies, providers and Blob rules. Removing a graph declaration
and applying is destructive: it deletes that graph's managed storage, including
all branches, retained history and managed Blob bytes. Review the deletion in
`cluster plan` and keep any required backup before applying. There is no separate
delete flag or unregister mode. External Blob source objects are not deleted.
On object stores, deletion removes the active namespace; provider version history
and retention policies may keep older objects. This is not secure erasure.
Live apply closes the affected graph to new requests and waits for its admitted
work and response bodies before deletion. Other graphs continue serving. If
deletion is interrupted, reconcile the original deployment ID; the recorded
operation resumes removal of the exact root. Editing the desired configuration
does not cancel accepted deletion, and absence during recovery does not create a
replacement graph.
Importing an existing root, recreating a missing managed graph, accepting schema
drift and repairing catalog payloads are not apply modes. Restore missing or
damaged authoritative data from a verified backup before deploying changes to
that graph.
Every present affected root is checked against its achieved schema identity,
even when source text changes. Removing an already-absent graph still records a
completed deletion; it never recreates the root. An out-of-band replacement or schema change refuses with
`applied_schema_drift`; matching text cannot authorize a different graph identity.
Plan always observes without taking the cluster writer lock. It reports the
existing owner and runs the same effect-free preparation used by apply, with
the intended actor. Its migration steps come from that preparation, not a second
schema-file read. Apply prepares again under writer admission because a preview
reserves nothing. Unavailable affected graphs, schema drift and migration
restrictions are errors, except that deletion accepts an already-absent root. An unavailable unrelated graph does not block an
independent change. Live apply also validates provider secrets and serving
settings on the server.
## Operational boundaries
* One mutation-capable process owns a cluster. Online deployment runs inside it,
and so can `optimize`, as a [recorded operation](https://www.omnigraph.dev/docs/operations/recorded-operations.md);
other direct maintenance requires an ownership handoff.
* Object-storage clusters may boot directly from `s3://bucket/prefix` or
`az://container/prefix`; source files are not needed for serving or recovery.
* Azure remains a qualification preview. Its running server retains the
mandatory [admission wrapper](https://www.omnigraph.dev/docs/getting-started/deployment.md#azure-blob-preview) during HTTP
deployment; submission does not acquire a second storage-writer lease.
* A schema drop removes data from the branch head without reclaiming storage.
Retained historical commits remain readable until cleanup removes them.
# Concepts
Source: https://www.omnigraph.dev/docs/concepts
OmniGraph is a typed property-graph database. A graph contains **nodes**,
**edges**, and a schema that defines their properties and constraints. The same
graph can be queried with structured graph patterns, full-text search, and
vector search.
## Graph model
* A **node type** describes an entity such as `Person` or `Document`.
* An **edge type** connects two node types, such as `WorksAt: Person -> Company`.
* A **property** has a declared type and can be nullable.
* Constraints such as keys, uniqueness, ranges, and edge cardinality are checked
on every write path, for every node and edge the write changes and every node
whose edges it changes (see
[edge cardinality](https://www.omnigraph.dev/docs/schema.md#edge-cardinality)).
Schemas use the [`.pg` language](https://www.omnigraph.dev/docs/schema.md). Reads and mutations use the
[`.gq` language](https://www.omnigraph.dev/docs/queries-search/queries.md).
## Consistency
Every query reads one consistent snapshot of the whole graph. A concurrent
write cannot become visible halfway through a query.
One mutation query or load publishes one graph commit. If it fails, none of its
changes become visible. Separate mutation queries are separate commits.
[Branches](https://www.omnigraph.dev/docs/branching.md) provide isolated, durable workspaces for
multi-step changes. Each change on a branch is still its own commit; merging the
branch makes the combined result visible on the target in one atomic step.
## History
Commits form a graph-wide history. You can:
* inspect commits and their actors;
* read a branch head or an earlier commit;
* compare two points in history;
* merge branches with entity-level conflict reporting.
History remains available until destructive cleanup removes the storage
versions it needs. Live branches can retain older versions, so delete branches
you no longer need.
## Storage and indexes
A graph is stored at one local, S3, or Azure Blob root. That root is the graph's
consistency and history boundary; cross-graph transactions are not supported.
See [storage](https://www.omnigraph.dev/docs/concepts/storage.md) for supported URI forms and backend configuration.
Indexes improve performance but do not define logical correctness. A declared
index may be missing or cover only part of recent data; queries still return
matching entities outside its coverage. Run `omnigraph optimize` to compact
data and refresh index coverage.
## Access paths
The embedded engine, CLI, and HTTP server share the same graph semantics. The
server adds bearer authentication and Cedar authorization; direct storage
access does not pass through server authentication.
* [Quickstart](https://www.omnigraph.dev/docs/getting-started/quickstart.md)
* [CLI guide](https://www.omnigraph.dev/docs/cli.md)
* [HTTP server](https://www.omnigraph.dev/docs/operations/server.md)
* [Clusters](https://www.omnigraph.dev/docs/clusters.md)
# Storage
Source: https://www.omnigraph.dev/docs/concepts/storage
One OmniGraph graph lives at one storage root. The root contains the graph's
schema, data, branches, and commit history. Treat it as a single unit: do not
edit files beneath it with storage-provider tools or write to its underlying
datasets directly.
## Supported graph URIs
| URI | Use |
| ----------------------------------------------------------------- | ---------------------------------------- |
| `/absolute/path/graph.omni` or `file:///absolute/path/graph.omni` | Local filesystem |
| `s3://bucket/prefix/graph.omni` | Amazon S3 or an S3-compatible service |
| `az://container/prefix/graph.omni` | Azure Blob Storage qualification preview |
An `http://` or `https://` URL addresses an OmniGraph server; it is not a graph
storage URI. These are engine/server storage capabilities, not CLI target variants.
Operator version 1 permits only local direct targets. Register a server or managed
target to access remotely served graphs, including servers using cloud storage;
see [operator configuration](https://www.omnigraph.dev/docs/cli/targets.md).
Azure support is a qualification preview: local Azurite validation and a live
managed-identity smoke deployment are complete, but the adversarial live-Azure
matrix is still pending. Every
mutation-capable Azure process must use the admission wrapper described in
[deployment](https://www.omnigraph.dev/docs/getting-started/deployment.md#azure-blob-preview).
## Consistency and history
Although a graph contains separate data for each node and edge type, OmniGraph
presents one graph-wide snapshot. A successful write makes all affected types
visible together. Readers never see half of a graph commit.
Renaming a type with `@rename_from` preserves its identity and history. Dropping
and later recreating a declaration starts a new lifetime, even when the public
name is reused.
`omnigraph optimize` rewrites physical layout without deleting history.
`omnigraph cleanup` is different: it permanently removes old storage versions,
which can make earlier commits unreadable. Review the retention policy and
backups before confirming it, and quiesce long-lived historical or Blob readers
first. An unconfirmed cleanup echoes the policy but does not inspect candidates.
Cleanup reclaims table versions, not the graph's own catalog. Every graph
commit adds a version of the branch's catalog, and no command removes the
versions of a live branch, so its storage grows with the number of commits;
plan storage for it on write-heavy graphs. A deleted branch's catalog can be
collected by `cleanup` once nothing else depends on it.
## Local filesystem requirement
Local read-write graphs require filesystem hard-link support. Read-only access
can work without it, but initialization and writes fail loudly when the
filesystem cannot provide the required create-if-absent behavior. Use a native
local filesystem rather than a mount that emulates files incompletely.
## S3 configuration
OmniGraph uses the standard AWS credential chain. Common settings include:
* `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and optional
`AWS_SESSION_TOKEN`;
* `AWS_REGION` or `AWS_DEFAULT_REGION`;
* `AWS_ENDPOINT_URL_S3` for an S3-compatible endpoint;
* `AWS_S3_FORCE_PATH_STYLE=true` when the endpoint requires path-style URLs;
* `AWS_ALLOW_HTTP=true` only for a trusted local development endpoint.
A graph's commit protocol relies on the store honouring conditional writes
(`If-None-Match` and `If-Match` on `PUT`), which the server requests on every
S3 binding. The project's S3 test suites run against RustFS; qualify any other
S3-compatible service before production. Google Cloud Storage's S3
interoperability endpoint is untested, and a store that ignores the conditions
can let two writers publish the same graph version. Native `gs://` URIs are not
supported.
## Azure configuration
Azure roots use the strict form `az:///`. Credentials never
belong in the URI.
* `AZURE_STORAGE_ACCOUNT_NAME` selects the account.
* `AZURE_STORAGE_CLIENT_ID` optionally selects a user-assigned managed identity.
* `AZURE_STORAGE_ENDPOINT` selects a custom Blob endpoint.
* `AZURE_STORAGE_USE_EMULATOR=true` enables Azurite; use
`AZURITE_BLOB_STORAGE_URL` to override its endpoint.
Azure Container Apps supplies its managed-identity endpoint variables at
runtime. Restart the process after changing storage-account, endpoint, or
identity selection.
For deployment, server boot, and cloud-specific safety requirements, see
[Deployment](https://www.omnigraph.dev/docs/getting-started/deployment.md).
## Encryption at rest
The graph's data, history and Blob payloads are ordinary objects in the storage
root, and nothing encrypts them on the way. Encryption at rest is the storage's:
enable default bucket encryption (SSE-S3 or SSE-KMS) on S3, the service's
equivalent on other S3-compatible stores, Azure Storage service encryption, or
an encrypted volume for a local root. There is no field-level encryption; keep
secrets out of graph properties, or encrypt them before they are written.
# Deployment
Source: https://www.omnigraph.dev/docs/getting-started/deployment
OmniGraph can keep a cluster on a local filesystem, S3-compatible object
storage, or Azure Blob Storage. The server always boots from one cluster root
and exposes its ready applied graphs under `/graphs/{graph_id}/…`; use
`--require-all-graphs` when any blocked graph must fail startup.
Servers use the v0.14 HTTP contract and open graph storage formats v14 to v17. The
cluster ledger has a separate version: explicitly converting it to v2 preserves
graph data and history. Start with [Operating a cluster](https://www.omnigraph.dev/docs/clusters.md) to
create and apply a deployment bundle.
## Binary
```bash
OMNIGRAPH_SERVER_BEARER_TOKENS_JSON='{"act-service":"secret"}' \
omnigraph-server \
--cluster /srv/omnigraph/company-brain \
--bind 0.0.0.0:8080
```
For object storage, pass the applied storage root:
```bash
AWS_REGION=us-east-1 \
OMNIGRAPH_SERVER_BEARER_TOKENS_JSON='{"act-service":"secret"}' \
omnigraph-server \
--cluster s3://company-data/omnigraph/company-brain \
--bind 0.0.0.0:8080
```
Use `GET /healthz` for process health and `GET /readyz` for readiness:
`/readyz` reports `loading` (startup is still pending), `serving`,
`degraded` (some graphs unavailable), `blocked` (none ready), or `draining`.
It includes the applied `config_digest` it booted from
(`booted_serving_digest`), the ledger revision, and `served_graph_count`,
`ready_graph_count`, `loading_graph_count` and `blocked_graph_count`.
`served_graph_count` counts the whole registry. Graphs closed for a transition also
contribute to `blocked_graph_count`. Readiness stays 503 while any graph is
loading. Once startup finishes, it returns 200 when at least one graph is ready
or the applied inventory is empty, and 503 when all graphs are unavailable or
shutdown has begun. The listener opens after configuration and
authorization validation, before graphs open. Healthy graphs become available
as they finish loading, while readiness holds aggregate traffic until loading
finishes. A nonempty cluster with no healthy graph exits with a
startup error after its opening attempts finish. With `--require-all-graphs`,
all graphs remain unavailable until every graph opens successfully; any failure
stops the server. Wait for readiness before sending data requests: the printed
listening address and `/healthz` confirm only that the listener is live.
Readiness is unauthenticated and names no graph. Authorized `GET /graphs`
returns one `graphs` list, including blocked entries, with each graph's `state`
(`loading`, `ready`, `blocked`, `transitioning` or `stopping`), `read_available`, `write_available` and
`action`. Availability describes the runtime, not the caller's permissions.
Blocked entries include a sanitized `failure`: `invalid_configuration`,
`invalid_policy`, `invalid_external_blob_policy`, `open_failed` or
`invalid_stored_queries`. Details remain in server logs.
`apply_correction_or_restart` means deploy corrected queries, policies or provider
configuration, or restore damaged storage from a verified backup and restart.
It never authorizes adopting a different graph identity. `wait_for_restart` means
shutdown. `wait_for_startup` means the single startup admission is incomplete;
it promises no retry time or additional open.
`wait_for_transition` means that graph's admissions are closed while its owner
finishes a transition. Wait for that owner, or restart if the transition was
abandoned or expired; there is no promised retry time. Other ready graphs can
continue serving and transitioning. Server-owned deployments activate schema and
stored-query changes through this transition; a proved pre-effect refusal restores
the unchanged views. See [live deployment](https://www.omnigraph.dev/docs/clusters.md#deploy-without-restarting).
Ready entries use `none`. There is no separate `quarantined` list or automatic startup retry.
`GET /cluster/deployments/{deployment_id}` reports `active` only when this process has installed
the exact deployment and all its graph bindings are ready. Loading, blocked graphs,
partial convergence and shutdown keep it false; a restart must verify its bindings
again. Successful apply alone is not activation, which never rewrites the durable
achieved result or receipt.
Known loading, blocked or transitioning graphs return 503 (`graph_unavailable`) to callers authorized to
read `main` or list the management inventory; other callers cannot use this response to discover
them. Unknown graphs return 404. A 503 does not authorize replaying a write.
`omnigraph graphs list --target NAME` displays graph ID, state and URI for a
registered server or managed target. `--json` preserves the full inventory
fields. Minimal discovery remains IDs/names only.
Admitted writes and reads with fully accepted request bodies retain capacity
until execution finishes, even after client disconnect. Writes include optional
source deletion after a merge. A lost response leaves the write's outcome
uncertain; follow [failure outcome](https://www.omnigraph.dev/docs/operations/troubleshooting.md#failed-data-write-commands)
before resubmitting. MCP cancellation and its 30-second response deadline stop
waiting; the admitted read continues and retains its tool slot until completion.
SIGTERM closes operation admission and stops accepting connections. The server
waits for entered graph opens, admitted writes, executing reads, read bodies and
stream producers for at most `--shutdown-grace-seconds` (else
`OMNIGRAPH_SHUTDOWN_GRACE_SECONDS`, else 25), then logs unfinished work and exits 2.
A thread enforces the deadline even if the runtime stalls. Signal handling starts
after configuration loading, before graph opening. Disconnected work remains in
the wait; a graph finishing startup during shutdown cannot become available.
A panic, or an uncertain outcome the engine cannot scope to one write's keys,
closes admission for every graph and starts the same shutdown without renewing
its deadline; a storage failure alone never does ([storage failures](https://www.omnigraph.dev/docs/operations/storage-failures.md)).
Once HTTP connections and other known logical owners finish, it exits 2
immediately; the watchdog is the upper bound. Unresolved reservations stay
charged until exit. This contains the process; it does not prove native I/O
settled or the engine can be reused.
A clean native shutdown exits 0. Set the orchestrator's own
termination grace longer than this value; a cutoff is crash-equivalent for the
work it interrupts, and the next open recovers it as after any crash.
On local storage and S3, the server also closes its storage scope, waits for
accepted graph and control writes, and releases its own cluster admission.
The next server can start normally with the same `--cluster` root. An initial
`--bootstrap-handoff` receipt is only for the first process; omit it on later
starts. The data, history and applied configuration remain in place.
Uncertain or interrupted work retains admission and exits nonzero; on S3 roots
a restarted server takes it over through the lock's lease
([restart after a failure](https://www.omnigraph.dev/docs/operations/storage-failures.md#restart-after-a-failure)). S3 release
uses a conditional marker, so a lost release response cannot delete a later
owner's lock. A nonclean exit is still a reason to inspect the original outcome,
not to replay writes. Azure keeps its existing shutdown behavior: even a clean
exit retains admission and requires the independent Azure recovery procedure.
If `OBJECT_STORE_CLIENT_MAX_RETRIES` is set, it must be `0` ([retries](https://www.omnigraph.dev/docs/operations/storage-failures.md#retries)).
A clean shutdown that leaves a reused key unsettled retains admission.
Other direct writers still follow the [ownership-transfer procedure](#writer-topology).
## Admission limits
Admission refuses excess operations and input immediately. Set these process-wide
environment variables before startup; sizes are integer bytes. Invalid numbers
warn and use the default; zero capacity refuses the corresponding lane.
| Variable | Default | Resource |
| ------------------------------------- | ------------------: | ---------------------------------------------------------------------------- |
| `OMNIGRAPH_GLOBAL_INFLIGHT_MAX` | 64 | Admitted write operations across actors |
| `OMNIGRAPH_GLOBAL_BYTES_MAX` | 268435456 (256 MiB) | Estimated retained write-input bytes across actors |
| `OMNIGRAPH_PER_ACTOR_INFLIGHT_MAX` | 16 | Admitted writes per actor |
| `OMNIGRAPH_PER_ACTOR_BYTES_MAX` | 4294967296 (4 GiB) | Estimated retained write-input bytes per actor |
| `OMNIGRAPH_ACTIVE_ACTORS_MAX` | 1024 | Actors with admitted writes; idle records are removed |
| `OMNIGRAPH_INGRESS_INFLIGHT_MAX` | 64 | Write requests retained by collection, operations or responses |
| `OMNIGRAPH_INGRESS_BYTES_MAX` | 268435456 (256 MiB) | Reserved/retained write-body bytes |
| `OMNIGRAPH_READ_INGRESS_INFLIGHT_MAX` | 64 | Read/MCP requests with bodies retained by collection, execution or responses |
| `OMNIGRAPH_READ_INGRESS_BYTES_MAX` | 67108864 (64 MiB) | Reserved/retained read/MCP body bytes |
| `OMNIGRAPH_BODY_TIMEOUT_SECONDS` | 30 | Time allowed to collect one complete request body |
The default ingress allowances total 320 MiB across the independent lanes;
configure both byte limits when setting an instance's input budget.
Body timeouts above 86400 seconds also warn and use the 30-second default.
Ingress reserves the route's maximum before reading, then reduces the reservation
to the actual body size. Body collection limits remain 1 MiB for ordinary JSON
requests, 32 MiB for bulk load and 64 KiB for MCP. Cluster deployment POSTs
allow 16 MiB plus 1 KiB for the request envelope; the captured bundle itself is
limited to 16 MiB. MCP uses the read-body lane after authentication. Only registered
bulk routes receive the larger limit; a stored query named `load` or `ingest` keeps the
ordinary JSON limit. Stored-query admission uses the registry's typed read/write
kind. Disconnect does not release capacity held by executing work; the body timeout
ends at complete collection and cannot cancel admitted execution. Input limits do
not bound process RSS or all engine memory and I/O. Size instances for graph,
workload and concurrency as well as request bytes.
The server allows 128 read-response observers and, independently, 64 write-response
observers. Executing reads, pending results, bodies, yielded bytes and their server
producers retain the slot until their final owner releases it. Bodyless reads
consume only a read observer; slow reads cannot exhaust write-body or write-response
capacity. Status routes bypass
ordinary admission so they remain callable during saturation.
Export and change-baseline responses reserve 256 KiB each from a fixed 2 MiB
process allowance, permitting eight simultaneous reservations. Each response
allows three outstanding 64 KiB chunks plus one pending producer chunk; at most
two chunks wait in its queue. Yielded chunks, including retained clones and
slices, keep their reservation until released. A new response waits at most
250 ms for process capacity before HTTP 413. Once admitted, a producer waits
for chunk capacity or disconnect without a per-chunk deadline; ordinary queue
backpressure has no additional deadline. An interrupted baseline supplies no
usable terminal cursor. Consumers must release each received server buffer to
continue; an in-process consumer retaining every buffer can exhaust its lane.
An HTTP client may still collect its received response: those client buffers are
separate from the server's transport buffers. These limits cover transport
payload buffers, not engine row/Arrow encoding, native work or total process RSS.
The engine separately limits retained keyed batches, keyed parse estimates
and removed-ID collections across each operation's tables; see
[mutation limits](https://www.omnigraph.dev/docs/queries-search/mutations.md#limits-and-conflicts). These fixed limits
return HTTP 413 with structured `resource_limit` details before the operation's
data fragments or publication, and leave the server available for smaller work.
They do not undo an implicitly created load branch or earlier command effects,
and do not establish a process memory ceiling or automatic retry safety.
## Container
The container entrypoint reads `OMNIGRAPH_CLUSTER` and binds to port 8080:
```bash
docker run --rm -p 8080:8080 \
-e OMNIGRAPH_CLUSTER=s3://company-data/omnigraph/company-brain \
-e AWS_REGION=us-east-1 \
-e AWS_ACCESS_KEY_ID \
-e AWS_SECRET_ACCESS_KEY \
-e OMNIGRAPH_SERVER_BEARER_TOKENS_JSON \
ghcr.io/modernrelay/omnigraph-server:
```
For a local cluster, mount the complete cluster directory and point
`OMNIGRAPH_CLUSTER` at the mount:
```bash
docker run --rm -p 8080:8080 \
-v /srv/company-brain:/var/lib/omnigraph/cluster \
-e OMNIGRAPH_CLUSTER=/var/lib/omnigraph/cluster \
-e OMNIGRAPH_SERVER_BEARER_TOKEN \
ghcr.io/modernrelay/omnigraph-server:
```
Terminate TLS at a load balancer or trusted reverse proxy. Keep bearer tokens
and storage credentials in the platform's secret store.
## Unix-socket access
`omnigraph-server` listens on TCP only. For clients with Unix socket support,
a gateway can relay a socket to its loopback listener:
```bash
socat UNIX-LISTEN:/run/omnigraph/server.sock,fork TCP:127.0.0.1:8080
```
The version-1 CLI does not accept Unix socket targets; use a registered HTTPS
endpoint instead. Relay requests and responses unchanged, including
`Omnigraph-Http-Api`. Restrict socket-directory permissions; bearer authentication
and policy still apply.
## S3-compatible storage
For AWS S3, configure the standard AWS credential chain and region. For a
compatible service, these variables may also be needed:
```bash
export AWS_ENDPOINT_URL_S3=https://objects.example.com
export AWS_S3_FORCE_PATH_STYLE=true
```
Set `AWS_ALLOW_HTTP=true` only for a trusted local development endpoint. Do not
put credentials in `cluster.yaml` or graph URIs. The store's requirements and
encryption at rest are in [Storage](https://www.omnigraph.dev/docs/concepts/storage.md#s3-configuration).
The same storage root must be visible to servers and out-of-band cluster or
maintenance jobs. Submit schema/query changes and graph additions to the running
server with `cluster apply --target`; the root's applied revision remains the
deployment artifact. Direct apply requires ownership transfer before serving.
## Azure Blob preview
Native `az:///` roots are implemented and tested with
Azurite. A live managed-identity smoke deployment is complete, but Azure
remains a qualification preview until the adversarial live-Azure matrix is
complete.
Every mutation-capable Azure process must be admitted under the cluster root:
```bash
AZURE_STORAGE_ACCOUNT_NAME=companygraph \
AZURE_STORAGE_CLIENT_ID= \
OMNIGRAPH_SERVER_BEARER_TOKENS_JSON='{"act-service":"secret"}' \
omnigraph-azure-admission run \
--mode server \
--root az://omnigraph/company-brain \
-- \
omnigraph-server \
--cluster az://omnigraph/company-brain \
--bind 0.0.0.0:8080
```
Use the same wrapper with `--mode job` and the same canonical `--root` for
bootstrap/apply jobs, `cluster upgrade-ledger`, root-addressed reconciliation,
`cluster force-unlock`, direct graph writers, and maintenance. Root-addressed
deployment status is read-only and does not need the lease. A v2 cluster's persisted
lock is additional admission; it does not replace the Azure lease or its
qualification boundary. The container entrypoint wraps an `az://` cluster automatically.
Replica-count settings are not a correctness fence: the admission lease is.
The checked-in [Azure reference deployment](https://github.com/ModernRelay/omnigraph/blob/main/deploy/azure/README.md)
contains the supported Container Apps topology, validation command, and
stuck-lease runbook. A stranded Azure lease and a retained v2 cluster lock are
separate: complete the lease recovery procedure before starting a wrapped
cluster-unlock or reconciliation job. Do not break a lease until the owner has
been identified and stopped.
## Writer topology
Run one mutation-capable writer process per cluster. This includes servers,
direct CLI writes, deployments, branch controls, and maintenance. On a v2
cluster, supported servers and direct CLI writers acquire the same exclusive
cluster admission before writable graph work and recheck the applied inventory.
CLI queries, exports, schema inspection/planning, lint and stored-query
validation check the applied identity without acquiring or changing that lock;
they refuse outstanding deployments or schema drift. Native maintenance
uses writer admission. An outstanding deployment
admits only reconciliation of its exact original ID.
Keep older binaries, raw storage tools and embedded writers outside this
cooperating boundary stopped; the lock cannot fence their native storage I/O.
Admission remains held after an ordinary direct write command succeeds or a
deployment completes. A clean server shutdown follows the bounded release
described above; ending work without that proof retains admission. A completed read-only
preflight refusal during new deployment, conversion or admission construction
releases its admission. Recovery of accepted work, cancellation and uncertain
effects retain it. A finished response, zero active HTTP
requests, lock age or a stopped PID alone does not prove that previously
accepted storage writes have settled. Before transferring ownership, stop the
prior owner and establish that its graph work and control-store I/O are terminal.
Exclude new admissions and other unlock attempts until the exact-ID unlock
finishes. Then start the next owner. Use the root-addressed
[status and recovery commands](https://www.omnigraph.dev/docs/clusters.md#inspect-and-recover-a-deployment)
to obtain the lock ID; unlocking never resolves an uncertain deployment.
Use the same current toolchain for restart. Older binaries refuse the released
S3 lock record; do not delete that record to bypass the version check.
A legacy v1 ledger refuses ordinary serving and writes. Stop its previous
owners and explicitly [convert the ledger](https://www.omnigraph.dev/docs/clusters.md#direct-deployments-and-conversion)
before using the current admission protocol.
Read replicas and zero-downtime overlapping writer replicas are not currently a
supported topology. Prefer stop-then-start replacement for a mutation-capable
server.
## Authentication and policy
Configure tokens from environment or a mounted secret file:
* `OMNIGRAPH_SERVER_BEARER_TOKEN`
* `OMNIGRAPH_SERVER_BEARER_TOKENS_JSON`
* `OMNIGRAPH_SERVER_BEARER_TOKENS_FILE` (mode 600, or `defaultMode: 0400` for a Kubernetes secret volume; a file other accounts can write is refused)
* `OMNIGRAPH_SERVER_BEARER_TOKENS_AWS_SECRET` in the AWS-enabled build
Policy bundles come from the applied cluster. See
[HTTP server](https://www.omnigraph.dev/docs/operations/server.md) and
[Authorization and actors](https://www.omnigraph.dev/docs/operations/policy.md).
## Upgrade and backup
Back up the whole cluster root, not selected physical files. Before a release:
1. read the release notes;
2. quiesce writers;
3. verify backups or exports;
4. upgrade the fleet together;
5. restart and run representative reads and writes.
Graph-format upgrades and cluster-ledger conversion are separate operations.
Follow the [storage upgrade guide](https://www.omnigraph.dev/docs/operations/upgrade.md) for qualified format
routes; `cluster upgrade-ledger` never resets or rewrites graph history.
# Install
Source: https://www.omnigraph.dev/docs/getting-started/install
## macOS and Linux
```bash
curl -fsSL https://raw.githubusercontent.com/ModernRelay/omnigraph/main/scripts/install.sh | bash
```
The installer downloads a release archive, verifies its SHA-256 checksum, and
installs the binaries in `~/.local/bin`.
Prebuilt archives support Linux x86\_64 and arm64, macOS arm64, and Windows
x86\_64. The Linux archives need glibc 2.39 or newer, as on Ubuntu 24.04 and
Debian 13. Build from source on other platforms, including Intel macOS.
Homebrew is also supported:
```bash
brew tap ModernRelay/tap
brew install ModernRelay/tap/omnigraph
```
### macOS signing and Keychain access
New macOS release archives are signed with EQTR INC's Developer ID and
notarized by Apple, on both stable and edge channels. The CLI keeps the
signing identifier `com.modernrelay.omnigraph` across updates so Keychain
can recognize the new executable as the same application.
When moving from an older ad hoc signed build, macOS may ask you once to
authorize the signed CLI to access an existing credential. Signing does not
grant access to an item you have not authorized. Source and Nix builds use
their own local signing identity and do not inherit release trust.
The release remains a tarball of three executables. Apple's notarization
ticket is checked online: a ticket cannot be stapled to a standalone CLI
binary or a tarball. A release can be published before Apple finishes
notarizing it; its binaries count as notarized once Apple accepts the
submission. Until then, macOS may refuse to open a copy downloaded with a web
browser. The install script downloads with `curl`, which does not mark files
for that check. Older published archives are unchanged.
## Nix
The repository's flake builds the three release programs from source with the
pinned Rust toolchain, the same packages and features as the release archives,
on Linux x86\_64 and arm64 and macOS arm64:
```bash
nix build github:ModernRelay/omnigraph
nix run github:ModernRelay/omnigraph -- version
```
Pin a commit or a release tag that carries the flake with
`github:ModernRelay/omnigraph/`. A build of a clean
checkout reports its commit; it reports no release channel.
## Windows
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -Command "iwr -UseBasicParsing https://raw.githubusercontent.com/ModernRelay/omnigraph/main/scripts/install.ps1 | iex"
```
Binaries are installed in `%USERPROFILE%\.local\bin`.
## Release channels
The default is the latest stable release; `install.sh` prints its tag before
downloading, and `install.ps1` names the channel. Only when no stable release publishes the platform's archive
(HTTP 404) does it fall back to `edge`, and it says so; a network or server
failure stops the install instead of switching channels.
Edge builds are prereleases built once a day from the tip of `main`, after its
whole CI run passes, when `main` has moved since the last one. Each is
published as its own release, `edge--` (for example
`edge-20261016-9c2b367`), whose files never change; the rolling `edge` release
always holds the newest one. The 30 newest edge builds are kept, and so is the
newest build of every storage format they no longer cover, since that build can
still export a graph written at that format. To install the newest:
```bash
curl -fsSL https://raw.githubusercontent.com/ModernRelay/omnigraph/main/scripts/install.sh \
| RELEASE_CHANNEL=edge bash
```
```powershell
iwr -UseBasicParsing https://raw.githubusercontent.com/ModernRelay/omnigraph/main/scripts/install.ps1 -OutFile install.ps1
powershell -NoProfile -ExecutionPolicy Bypass -File .\install.ps1 -ReleaseChannel edge
```
To deploy an edge build, pin it by its tag, so every machine installs the same
binaries and you can return to them: `VERSION=edge-20261016-9c2b367` on
macOS/Linux or `-Version edge-20261016-9c2b367` on Windows.
Each edge build is also a [container image](https://www.omnigraph.dev/docs/getting-started/deployment.md#container) with the
same tag, `modernrelay/omnigraph-server:edge-20261016-9c2b367` on Docker Hub
and `ghcr.io/modernrelay/omnigraph-server:edge-20261016-9c2b367`, holding that
build's Linux x86\_64 binaries on Debian trixie. The `edge` image tag follows the
rolling `edge` release; `latest` stays on the newest stable release. Pin a
deployment to the image's digest, which the run's summary lists. Provenance is
attached to the GHCR digest; do not use a Docker Hub manifest digest to verify
that attestation. GHCR access may require registry authentication; Docker Hub
is the public pull channel. See [container verification](#verify).
Read the build's release description before pointing it at a graph that a
stable release must still read. It lists the storage format the build writes
and the HTTP contract it speaks next to the latest stable release's. A branch
the build writes to is left at the build's storage format, and a stable binary
refuses a format newer than its own, so test on a copy of the graph or back it
up first ([upgrades](https://www.omnigraph.dev/docs/operations/upgrade.md)). A client and a server must speak
the same HTTP contract.
Install a specific stable tag with `VERSION=v0.14.0` on macOS/Linux or
`-Version v0.14.0` on Windows. Set `INSTALL_DIR` or `-InstallDir` to choose a
different destination.
Documentation on `main` may describe behavior newer than the latest stable
binary. Check `omnigraph version` and the [release notes](https://github.com/ModernRelay/omnigraph/blob/main/docs/releases/).
## Client compatibility
Upgrade the CLI, server, and HTTP integrations together. OmniGraph 0.14 requires
HTTP contract `0.14`; the npm `0.13.0` clients are not compatible with it.
Both `@modernrelay/omnigraph@0.14.0` and
`@modernrelay/omnigraph-mcp@0.14.0` are published on npm. Install the 0.14.0
clients alongside the 0.14 server; see the
[SDK release](https://github.com/ModernRelay/omnigraph-ts/releases/tag/v0.14.0).
For an existing deployment, follow the
[0.13 → 0.14 checklist](https://www.omnigraph.dev/docs/operations/upgrade-0.14.md), including configuration
and command replacements.
You can use the released CLI, the [HTTP API](https://www.omnigraph.dev/docs/operations/server.md), or the
server's built-in [MCP endpoint and CLI stdio bridge](https://www.omnigraph.dev/docs/operations/mcp.md)
without either npm package. The MCP protocol has its own negotiation; refresh
cached tool schemas when upgrading a client.
## Build from source
Install the Rust stable toolchain and the Protocol Buffers compiler (`protoc`),
then:
```bash
RUSTFLAGS= cargo build --release --locked \
-p omnigraph-cli \
-p omnigraph-server \
-p omnigraph-azure-admission
```
The empty `RUSTFLAGS` replaces the workspace `.cargo/config.toml` build flags,
which exist for the development test suites; release binaries build without
them.
Or use the source installer:
```bash
curl -fsSL https://raw.githubusercontent.com/ModernRelay/omnigraph/main/scripts/install-source.sh | bash
```
## Installed programs
Release archives contain:
* `omnigraph` — CLI;
* `omnigraph-server` — HTTP server;
* `omnigraph-azure-admission` — Azure writer-admission utility.
The admission utility's `inspect`, `break`, and help commands are available on
Windows. Its child-supervision `run` mode is supported only by the Unix
deployment images used for the Azure reference topology.
## Verify
```bash
omnigraph version
omnigraph-server --help
omnigraph-azure-admission --help
```
`omnigraph version` reports what the binary itself speaks, without opening a
graph or contacting a server. For example, building the `v0.14.0` source tag
without release metadata prints:
```text
omnigraph 0.14.0
internal-schema 16 (serves v14 to v16)
http-contract 0.14
```
`omnigraph version --json` also reports the machine-readable CLI protocols:
```json
{
"version": "0.14.0",
"channel": null,
"commit": null,
"http_contract": "0.14",
"protocols": { "studio": 1, "target": 2, "credential": 2, "access": 2 },
"internal_schema": {
"writes": 16,
"serves": [14, 16]
}
}
```
An edge build reports a SemVer pre-release of the package version, such as
`0.15.0-edge+9c2b367`. Edge builds are published only while the package version
is above the latest stable release, so an edge version sorts after every
published stable release and before the release it precedes. `channel` is `stable`
or `edge` for a GitHub release archive and null for any other build. `commit`
is the commit a packaged build was compiled from; a plain source build reports
it as null and prints no `commit` line. The server reports the same commit as
`source_version` on `GET /healthz`.
`internal_schema.writes` is the storage format of a graph this binary
creates, and the one `omnigraph upgrade` converts a graph to; `serves` is the
range of formats it opens. A graph it opens keeps its own format, apart from
the v14 and v15 stamps described in [upgrading](https://www.omnigraph.dev/docs/operations/upgrade.md).
`http_contract` is the `Omnigraph-Http-Api` value the CLI sends. A server
accepts only an exact match and returns its own value on every response,
including `HEAD /healthz` ([HTTP contract](https://www.omnigraph.dev/docs/operations/server.md#http-contract)).
When several CLIs are installed, for example one in `~/.local/bin` and an
older one from Homebrew, choose by this field, not by the package version.
`protocols.target` and `protocols.credential` identify compatible generations of
operator configuration/target administration and credential commands. Both are
2 for profiles, targets with one inline `auth` provider, and credential sources.
Match these numbers exactly when selecting a CLI for those contracts; they are
independent of the operator document's `version: 1`. Protocol 1 was published
with multiple incompatible 0.14 edge grammars, so that number alone cannot
select one of them. Use protocol 2 for the [current format](https://www.omnigraph.dev/docs/cli/targets.md).
An incompatible change to either contract requires its discovery number to change.
`protocols.access` identifies the `omnigraph access --json` shape: protocol 2
uses `actor_id` in place of protocol 1's `actor`. `protocols.studio` identifies
the [Studio link](https://www.omnigraph.dev/docs/cli/studio.md#the-link), `studio --json`, and the `home` and
`binding` digests `target inspect --json` reports for checking a link.
`internal_schema.writes` is the storage-format stamp of a graph this binary
creates, the highest it writes, and `internal_schema.serves` is the inclusive range, lowest first, that
normal open accepts; `snapshot --json` reports a graph's own stamp as
`internal_schema_version`. `omnigraph --version` (`-v`) prints only the first
line.
Release archives from v0.14.0 on, and every `edge--` build, carry
a build provenance attestation. When the GitHub CLI (2.49 or later) is installed
and signed in, both installers verify it after the checksum and refuse an
archive that fails; otherwise they say the provenance was not checked.
`VERIFY_ATTESTATION=require` refuses to install without the check,
`VERIFY_ATTESTATION=off` skips it. An explicitly requested `VERSION` older than
v0.14.0 has no attestation and installs with a warning. To check an archive
yourself:
```bash
gh attestation verify omnigraph-linux-x86_64.tar.gz --repo ModernRelay/omnigraph
```
The [container image](https://www.omnigraph.dev/docs/getting-started/deployment.md#container) carries the same stamp from
v0.14.0, and its GHCR digest carries the attestation:
```bash
docker run --rm --entrypoint /usr/local/bin/omnigraph \
modernrelay/omnigraph-server: version
gh attestation verify oci://ghcr.io/modernrelay/omnigraph-server: \
--repo ModernRelay/omnigraph
```
Continue with the [quickstart](https://www.omnigraph.dev/docs/getting-started/quickstart.md).
# Quickstart
Source: https://www.omnigraph.dev/docs/getting-started/quickstart
This walks the core loop end to end: define a schema, initialize a graph, load
data, query it, and use a branch. It uses a local file-backed graph. CLI direct
targets support local files only; access hosted graphs through a registered
server or managed target. See [operator configuration](https://www.omnigraph.dev/docs/cli/targets.md).
[Install](https://www.omnigraph.dev/docs/getting-started/install.md) the `omnigraph` CLI first.
The example builds a small evidence graph in which sources support a claim.
## 1. Write a schema
A schema (`.pg`) declares your node and edge types. Save this as `schema.pg`:
```graphql
node Source {
slug: String @key
title: String
}
node Claim {
slug: String @key
statement: String
}
edge Supports: Source -> Claim
```
See the [schema language](https://www.omnigraph.dev/docs/schema.md) for types, constraints, and edges.
## 2. Initialize the graph
```bash
omnigraph init --schema schema.pg graph.omni
```
`init` creates an empty graph at the given URI with your schema applied.
## 3. Load data
Data is newline-delimited JSON, one node or edge per line. Save this as
`evidence.jsonl`:
```jsonl
{"type":"Claim","data":{"slug":"lower-latency","statement":"The migration reduced request latency."}}
{"type":"Source","data":{"slug":"load-test","title":"Load test report"}}
{"type":"Source","data":{"slug":"production-metrics","title":"Production metrics"}}
{"edge":"Supports","from":"load-test","to":"lower-latency"}
{"edge":"Supports","from":"production-metrics","to":"lower-latency"}
```
The keyed `slug` values become node IDs, so the edges can refer to
`load-test`, `production-metrics`, and `lower-latency`. One load can contain
several node and edge types and publishes them as one graph commit. `--mode` is
required (`overwrite | append | merge`):
```bash
omnigraph load --data evidence.jsonl --mode overwrite --target ./graph.omni
```
For finer-grained writes and the node/edge JSONL shapes, see
[mutations and loading](https://www.omnigraph.dev/docs/queries-search/mutations.md).
## 4. Query
Write a query (`.gq`) — save as `queries.gq`:
```graphql
query sources_for_claim($claim: String) {
match {
$source: Source
$claim_node: Claim { slug: $claim }
$source supports $claim_node
}
return { $source.title as source }
order { source asc }
}
```
Run it:
```bash
omnigraph query sources_for_claim --query queries.gq \
--params '{"claim":"lower-latency"}' --format table --target ./graph.omni
```
This returns `Load test report` and `Production metrics`.
The query name is positional; `--query` points at the `.gq` source and
`--target` addresses the graph's storage directly.
The [query language](https://www.omnigraph.dev/docs/queries-search/queries.md) covers `match`/`return`/`order`, and
[search](https://www.omnigraph.dev/docs/queries-search/search.md) covers vector and full-text search.
## 5. Work on a branch
Branches isolate changes until you merge them — Git-style, across the whole
graph. Save an additional source and evidence link as `benchmark.jsonl`:
```jsonl
{"type":"Source","data":{"slug":"independent-benchmark","title":"Independent benchmark"}}
{"edge":"Supports","from":"independent-benchmark","to":"lower-latency"}
```
`branch` commands reserve their positional argument for the branch name, so
they address the graph with `--target` (the same flag `query` uses above):
```bash
omnigraph branch create review/add-benchmark --target ./graph.omni
omnigraph load --data benchmark.jsonl --mode append \
--branch review/add-benchmark --target ./graph.omni
omnigraph query sources_for_claim --query queries.gq \
--params '{"claim":"lower-latency"}' \
--branch review/add-benchmark --target ./graph.omni
omnigraph branch merge review/add-benchmark --into main --target ./graph.omni
```
See [branches & commits](https://www.omnigraph.dev/docs/branching.md) and [merging](https://www.omnigraph.dev/docs/branching/merge.md).
## Next steps
* [CLI reference](https://www.omnigraph.dev/docs/cli/reference.md) — every command and flag.
* [Schema language](https://www.omnigraph.dev/docs/schema.md) and [query language](https://www.omnigraph.dev/docs/queries-search/queries.md).
* [Operating a cluster](https://www.omnigraph.dev/docs/clusters.md) and [running the server](https://www.omnigraph.dev/docs/operations/server.md)
for multi-graph, multi-user deployments.
# Maintenance
Source: https://www.omnigraph.dev/docs/operations/maintenance
OmniGraph provides four direct-storage maintenance commands:
* `optimize` compacts data and reconciles declared indexes on `main`.
* `index build` brings one branch's declared indexes up to its rows.
* `index rebuild --kind full-text` replaces full-text indexes on one branch.
* `cleanup` permanently removes unretained table versions and unused table
forks left by branches created before storage format v11.
They do not run through the HTTP server; a cluster server runs optimize as a
[recorded operation](https://www.omnigraph.dev/docs/operations/recorded-operations.md). Address a graph directly, or from a cluster root:
```bash
omnigraph optimize --target ./graph.omni
omnigraph optimize --target ./cluster-root --graph knowledge
```
Follow the command-specific concurrency requirements below. Registered direct
targets accept local files only; an inline [S3 cluster root](https://www.omnigraph.dev/docs/operations/s3-cluster-roots.md)
is the one cloud-direct exception. See
[operator configuration](https://www.omnigraph.dev/docs/cli/targets.md#transport-and-capability-boundaries).
Each command acts as its `--as` actor, and the commit it publishes records
that actor. On a cluster graph the CLI installs the policy the cluster applied
to it, and an embedded host may install its own; either checks the
[policy actions](https://www.omnigraph.dev/docs/operations/policy.md#actions) before any effect: `maintain` for optimize,
index builds and the full-text rebuild, and `reclaim` for cleanup. A standalone
graph has no policy. The actor is self-declared, so storage permissions remain
the trust boundary.
## Optimize
```bash
omnigraph optimize --target ./graph.omni
omnigraph optimize --target ./graph.omni --json
```
Optimize rewrites small fragments into fewer larger fragments, rewrites every
fragment that still stores a dropped property's values, rebuilds each scalar or
vector index whose coverage lags behind appended rows, and builds missing
declared indexes that are ready to build. It works on `main`. `index build`
does only the index part, on any branch including `main`; compaction, the
dropped-value rewrite and the traversal artifact stay with optimize. Optimize
does not delete old versions or collect unused table forks. Use `cleanup` for
storage reclamation.
Dropping a property removes it from the schema without rewriting its table (see
[schema changes](https://www.omnigraph.dev/docs/schema.md#schema-changes)), so its values stay in the
table's data files. Every optimize rewrites each fragment that still holds them,
whatever its size, copying only the remaining properties; the dropped values are
not read. A `cleanup` then deletes the old files, Blob values included, once
nothing it retains references them: no commit its retention keeps, no live
branch created from a commit before that optimize, and no tag naming such a
commit. Merging a branch does not release its files; deleting it does. A table
holding no dropped values and nothing else to compact is left alone.
Each table's work is staged as detached Lance versions of the table's current
pin and published in one graph commit, like any other write. A run that fails
before that commit leaves the graph unchanged and no recovery state; after it
the published pins are the tables' versions, with nothing left to finish.
Optimize runs beside live writers: a write that lands on a table while its
compaction is staged fails the run with a read-set conflict, and the next run
re-plans from the new state. A write prepared before an optimize published goes
ahead, or prepares again once, holding writes off, if it changes a row the run
moved; none reports a conflict for it. Lance compacts neighbouring fragments only when the
same indexes cover them, so a table whose index coverage was uneven before a run
may coalesce fully only on the next run, after the rebuilt coverage is in place.
Compacting a table with Blob properties reads each managed Blob value into
memory. Optimize sizes each compaction batch from its largest row, whatever
`LANCE_DEFAULT_BATCH_SIZE` says, so one batch holds at most 32 MiB of managed
Blob payload; a larger row is read whole, external references are not read,
and the writer's own copies come on top. A Blob descriptor the decoder rejects
refuses the compaction before that table is rewritten. Optimize plans up to
`OMNIGRAPH_MAINTENANCE_CONCURRENCY` tables ahead (default 8) and rewrites up to
that many at once, with that many compaction tasks, index folds and builds in
all (a Blob table's tasks one at a time): budget for that many Blob tables.
### Bounding a run
Writes to `main` wait while optimize holds its writer gates. A run given
`--max-gate-hold 30s` starts no table after that long except the first: the
rest are `deferred`, finished tables publish, and a writer waits at most the
bound plus one rewrite. Tables run in type-key order; `resume_at` names the
first deferred one, and `--tables` runs only the listed tables, in order. Each
table's `outcome` is `optimized`, `unchanged`, `deferred` or
`failed` (with its `error`; the other tables still publish). The report adds
`input_commit`, `graph_commit_id`, `gate_hold_ms` and the `configuration_clear`
and `artifact` outcomes, which never fail a published run.
Optimize also persists the traversal-adjacency artifact
(`__graph_index/csr-current.bin`), which cold traversal builds load instead of
scanning every edge dataset. It is derived: every load is verified and falls
back to the in-memory build, so a stale copy or a failed write (the `artifact`
outcome) costs only its speedup. Optimize builds it after releasing the writer
gates, and only when the run advanced an edge dataset or no artifact exists; a
build skipped for its size is counted again, up to the limit, by each such run.
Existing full-text indexes are preserved rather than incrementally merged.
Unindexed rows remain searchable, but a growing uncovered tail can cost more to
scan. Use `index rebuild --kind full-text` to refresh that coverage or migrate
an old analyzer generation. Optimize reports uncovered full-text coverage under
`pending_indexes` with this remedy. Deferred coverage alone creates no graph
commit or maintenance work. A vector index whose property has no usable vectors
also stays pending; run optimize again after loading or generating vectors.
## Build indexes
```bash
omnigraph index build --target ./graph.omni --branch review
omnigraph index build --target ./cluster-root --graph knowledge \
--branch review --json
```
On any branch, `index build` builds every declared index a table lacks, such as
those of a type whose first rows were loaded there, and rebuilds whole each
BTREE or vector index that leaves rows uncovered. Every table publishes in one
graph commit on that branch; other branches and snapshots keep their indexes,
and a branch with nothing to do publishes nothing. The default branch is `main`.
As with optimize, a write that lands on a table while its batch is staged fails
the run with a read-set conflict; run the command again. A vector property with
no vectors yet and full-text coverage stay pending: only `index rebuild --kind
full-text` refreshes a full-text index's coverage, and queries match uncovered
rows with the index's analyzer meanwhile.
JSON reports `branch`, `graph_commit_id` (null for a no-op), `indexes` with each
`type_key`, `property` (the indexed column, `__id` for the identity index),
`kind` (`btree`, `full_text` or `vector`) and `action` (`built` or `refreshed`),
and `pending_indexes` with each `type_key`, `property` and `reason`.
## Rebuild full-text indexes
```bash
omnigraph index rebuild --kind full-text --target ./graph.omni --branch main
omnigraph index rebuild --kind full-text --target ./cluster-root \
--graph knowledge --branch review --json
```
`--kind` is required. The deprecated spelling `rebuild-full-text-indexes` still
works for one release. Under a policy, the rebuild needs `maintain` on the
selected branch.
The command fully replaces text-search indexes from current entities using the
engine's default English analyzer, including already indexed entities. It builds
declared node full-text indexes and replaces existing physical full-text indexes
on nodes or edges; it does not create edge-property indexes from declarations.
External custom tokenizer settings are not preserved. Completed rebuilds warn
on stderr, or in the JSON `warnings` array, that those settings were replaced.
All rebuilt datasets become visible in one graph commit. JSON reports `branch`,
`graph_commit_id`, and `rebuilt_indexes` with each `type_key` and `property`.
Success means the selected branch's planned rebuild was published, or an explicit
no-op with an empty index list, null commit, and empty `warnings` array. The default
branch is `main`.
Other branches and historical snapshots are not rewritten. Rebuild every live
branch that needs full-text search; restoring an older snapshot may require
rebuilding again. The operation does not regenerate embeddings or alter entity
values. Stop overlapping writers and preserve a backup before an upgrade; see
[full-text upgrades](https://www.omnigraph.dev/docs/operations/upgrade.md#full-text-index-upgrade).
Unknown legacy or external index kinds require a controlled migration, not a
guessed replacement; see [unsupported inventory](https://www.omnigraph.dev/docs/operations/upgrade.md#unsupported-index-inventory).
## Cleanup
Cleanup is a tracing collector over the graph's table storage. Every table write
is a detached Lance commit that a graph commit pins, so what a table keeps is
decided by which graph commits the run retains. Per live branch the run computes
the graph commits its policy keeps, takes the table versions those commits pin
as roots, marks every file the root manifests reference (base and overlay data
files, deletion files, index directories, transaction files), and deletes
unretained table versions and the files only they reference. Selected merge
bases and native tagged snapshots are retained in addition to the versions
selected by the policy. Historical table lifetimes participate even after a type
is dropped or its name is re-added under a new identity. Blob sidecars follow
their parent data file's reachability.
`omnigraph cleanup` is the only cleanup to run against a graph root. Lance's
own cleanup (`cleanup_old_versions`, and every vacuum or retention tool built
on it) does not trace OmniGraph's detached table versions. On Lance 13 it can
delete files used only by detached versions once those files are older than
seven days, leaving the manifests present but affected snapshots unreadable.
This can affect the current graph or a recent commit that inherits an old file;
files shared with retained attached versions remain protected. The same risk
applies to the graph's catalog dataset, whose unpublished proposals and
unattached snapshots are detached versions. If an external cleanup has run,
stop writes and check for missing files; restore the whole root from a verified
backup if data was deleted.
A table version no graph commit ever named is unpublished staging: a write
in flight, or one that failed before its publication. Unless a native tag
protects it, it is deleted when the branch incarnation and graph head its
commit recorded can no longer be published against: its valid incarnation is absent from a complete inventory taken
after the staged manifests were listed, or the captured head moved past its
publication without naming it. Missing or malformed ownership is kept.
Cleanup validates its complete branch and tag capture after all table
inventories; an overlapping change refuses the plan before deletion. Cleanup also removes table forks
left by branches created before storage format v11 once nothing references
them and the graph branch incarnation in their name is gone. Branch deletion
and later writes leave reclamation to this command. Without `--confirm`, the
CLI only echoes the requested retention policy and exits before opening the
graph; it does not enumerate candidates:
```bash
omnigraph cleanup --keep 10 --older-than 7d --target ./graph.omni
```
Run the reviewed policy with `--confirm`:
```bash
omnigraph cleanup --keep 10 --older-than 7d --confirm --target ./graph.omni
```
At least one retention option is required:
| Option | Meaning |
| ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--keep N` | Retain the newest `N` graph commits of every live branch, and every table version they pin |
| `--older-than DURATION` | Also retain every graph commit newer than the cutoff, on every live branch; and collect a pre-v11 table fork only once every data and branch-reference object of the fork is older than the cutoff |
A graph commit survives when either option retains it. The current HEAD of
every live branch is always retained, and a named branch also keeps its
oldest commit, the copy of the parent's commit it was created from, while
the branch lives; for a branch started at a past commit, that is the commit
it names. A commit cleanup no longer retains starts no branch
([start a branch at a commit](https://www.omnigraph.dev/docs/branching.md#start-a-branch-at-a-commit)).
Cleanup also retains the selected merge bases between
current branch heads and accepted merge inputs. Native tags protect the exact
snapshots they name. A tagged graph snapshot retains all its table versions; a
tagged table snapshot retains that version, including a detached version or a
version on main.
`--keep` counts graph commits on that branch, never Lance versions of a dataset: `--keep 10` keeps the
last ten graph commits of each live branch readable, with every table version one of them pins, whatever
their age. A pre-v11 fork that no registration references is collected once the graph branch incarnation
in its name is gone: immediately with `--keep` alone, and with `--older-than` only after every object of
the fork is older than the cutoff. Native tags, readable graph snapshots and required native ancestors
also protect forks. Choose a policy that matches your rollback and audit needs. Planning lists each live
branch's graph commits. `--keep` alone reads only the commits it retains and the merge bases, so an
unreadable commit it drops no longer stops it; `--older-than` also reads the time of every commit not
otherwise retained, up to 64 at once on object storage, and stops on one it cannot read.
Before cleanup:
1. stop long-lived Blob readers and readers of snapshots the policy removes;
2. verify important branches and snapshots;
3. make or verify a backup/export;
4. review the retention policy against the snapshots and branches you still
need;
5. review the exact retention command and confirmation target.
Cleanup fails closed per table: a trace that does not finish (a pinned
version missing from the table's version listing, or a read that failed)
deletes nothing for that table and reports why in its result row; fix the
cause and rerun cleanup to converge. Cleanup deletes a table version's unused
files before the version itself. If a run stops between the two steps, the
next run finds that version again, counts the files already gone as removed,
and deletes the version. The run derives each branch's history
and pins from one captured version, then validates that the live inventory,
versions, incarnations and tags still match after all table inventories.
A changed observation refuses the whole plan before deletion; rerun with the
same policy. This includes a branch deleted after cleanup listed it: cleanup
returns a retryable conflict, and a new run captures the current branches.
Retirement archives exact branch identity and ancestry inside
its native tree, then removes the active ref. Creation does not scan retired
histories. Cleanup removes unneeded retired trees and their archives, while
preserving trees needed by live descendants, merge bases or tags.
Ordinary graph writes and branch merges may overlap cleanup. A merge protects
its accepted source, target and base snapshots with durable native tags;
source advancement does not revoke those inputs. A cancelled merge or one
whose publication returns an in-doubt error keeps its tags until the target
witness proves it can no longer publish. An unchanged or unresolvable target
keeps them. Graph-branch creation/deletion and cleanup retain the existing
single-writer-process control boundary.
Legacy native snapshots can borrow another location's files. Cleanup retains
those physical origins, including overlay and index files. An incomplete origin
trace prevents physical-file deletion across the run. Unregistered native trees
can retain extra origin files until a later cleanup after those trees vanish.
Ordinary writes retain unpublished staging, because the branch head its commit recorded is still the
branch's head. A write that publishes during the run either landed before
the snapshot, in which case its version is a root, or after it, in which
case its staging is kept for the same reason. Staging left by a write that
failed before publication is kept until that branch publishes again or is
deleted; on a branch that never publishes again it stays, and its table's
result row counts it under `unpublished_manifests`. Age alone never decides
anything: an old staging is not proof that its writer stopped, and an old
pinned version is never removed while a retained graph commit names it.
Files no listed manifest references follow Lance's seven-day unverified-file
rule, including temporary manifests. Each orphan Blob sidecar must satisfy
that age rule itself, even when its parent data file is older.
Each table's result row (under `--json`) carries `bytes_removed`,
`manifests_removed` (table versions deleted: published ones no retained
graph commit names, and dead staging), `unpublished_manifests` and
`unpublished_bytes` (the staging the run found, dead ones included),
`foreign_versions` (Lance versions above the table's last linear version,
which the run never deletes) and `error`, set only when the table's trace
did not finish, in which case its counts are zero. `old_versions_removed`
stays for one release beside `manifests_removed` and carries the same
number; nothing defers a table, so the row has no `deferred` field.
Exit 0 means every table was visited and that nothing a retained graph
commit pins, and no file such a commit references, was removed. It does not mean every table was collected: read the result rows
for tables with an `error`.
### Foreign versions
Every Omnigraph table write is a detached Lance commit that a graph commit
pins, so a table's linear Lance history ends where the table was created. The
registration records that point as its last linear version
(`omnigraph.last_linear_version`). A linear commit above it came from
something other than Omnigraph writing the table directory, such as a Lance
client pointed at it. No read or write of the graph resolves the linear
HEAD, so a foreign version changes no query result and blocks no writer, and
no command adopts it. A confirmed cleanup run (`cleanup --keep N --confirm`,
which also removes what its policy does not retain) lists these versions per
table, under `foreign_versions` with `--json` and by name in the human report,
and leaves them and their files in place. The preview without `--confirm`
does not open the graph and lists nothing. Embedded hosts can read the same
list from the read-only `Omnigraph::cleanup_plan`. To discard them,
export the graph and load it into a new one; see
[Troubleshooting](https://www.omnigraph.dev/docs/operations/troubleshooting.md#foreign-versions).
## Suggested cadence
* Run `optimize` after large loads or on a regular cadence for write-heavy
graphs.
* Run `cleanup` from an explicit retention policy after backups and rollback
requirements have been reviewed.
Moving a graph to another storage format is not maintenance. See
[Upgrading](https://www.omnigraph.dev/docs/operations/upgrade.md).
# MCP
Source: https://www.omnigraph.dev/docs/operations/mcp
`omnigraph-server` serves the Model Context Protocol at `/mcp`, so Claude,
ChatGPT, Cursor and other MCP clients can work with a graph with a URL and the
caller's own credential. MCP shares the server's listener and lifecycle; no
separate service is needed.
Every tool calls the same handler as its HTTP route, under the same actor and
the same [policy](https://www.omnigraph.dev/docs/operations/policy.md) action. A tool appearing in the list is not
permission to use it: a call the policy refuses returns the route's refusal.
## Enable it
`/mcp` is mounted whenever the server has a bearer credential source: static
tokens, signed data tokens, or an
[OIDC resource identity](https://www.omnigraph.dev/docs/operations/server.md#oidc-resource-identities-and-mcp)
(`--oidc-identity-trust`). A server without credentials has no MCP.
| Credentials | `/mcp` | OAuth discovery | Challenge on a 401 |
| -------------------------------------------- | ------- | --------------- | ---------------------------------- |
| Static or signed data tokens | mounted | none | `Bearer` |
| OIDC resource identity, alone or with tokens | mounted | served | `Bearer resource_metadata=""` |
With static or signed tokens, configure the client with the URL and an
`Authorization: Bearer ` header; Claude Code and Cursor support this
directly. With OIDC, the resource identifier in the trust file is the URL users
enter in their MCP client, for example `https://graphs.example.com/mcp`, and the
client signs in through OAuth. With an identifier naming the service root,
clients that accept a path prefix can also sign in from other routes; see
[Sign-in and discovery](#sign-in-and-discovery). A client that can only start a
command uses [`omnigraph mcp`](#connect-a-stdio-agent).
The endpoint is stateless Streamable HTTP built on the official Rust MCP SDK:
each request is a `POST` that carries its own bearer credential and gets a JSON
response. There is no MCP session, and a successful `initialize` never admits a
later unauthenticated request.
### Upgrade tool schemas
After upgrading the server, reconnect clients and refresh cached tool schemas
with `tools/list`. The `access` tool now returns `actor_id` instead of `actor`;
clients with fixed response types must update those types before reconnecting.
Its output matches the [HTTP access report](https://www.omnigraph.dev/docs/operations/policy.md#learn-your-access).
`queries` now lists stored mutations too, marked `mutation: true`, and
`mutate` no longer requires `query`.
MCP does not use the `Omnigraph-Http-Api` header, so an old MCP client is not
automatically refused.
MCP protocol negotiation is unchanged, and the old field has no alias.
### Sign-in and discovery
A request without a valid credential is refused with 401 before its body is
read, with the same [challenge](https://www.omnigraph.dev/docs/operations/server.md#authentication) as every other route.
With OIDC, the challenge names the protected-resource metadata:
```text
WWW-Authenticate: Bearer resource_metadata="https://graphs.example.com/.well-known/oauth-protected-resource/mcp"
```
When a bearer credential was presented and refused (bad signature, expired,
wrong audience or organization), the challenge also carries
`error="invalid_token"`, which tells a client to refresh or sign in again.
Without OIDC the challenge is `Bearer`, which advertises no sign-in flow.
Two refusals of a valid token carry no challenge, because signing in again
issues a token for the same person and cannot help:
* the admission file names no principal for the token's subject: 403
`not_admitted`. The server's operator must admit the identity;
* the admission snapshot has expired, for example during a publisher outage:
503 `service_unavailable` with `Retry-After`. Access returns with the next
publication.
A token's `client_id` is kept beside the actor, which stays one `principal:`
for every client a person uses. The [access route](https://www.omnigraph.dev/docs/operations/policy.md#learn-your-access)
and the `policy decision` log line name it; commits do not yet. It is opaque
client-chosen text: render it escaped.
The server has one resource identifier, so every route's challenge names the
same metadata. RFC 9728 §3.3 has a client that followed a challenge require the
metadata's `resource` to equal the URL it requested, so a strict client uses the
metadata only on the URL the identifier names. Some clients accept an identifier
whose path is a prefix of that URL, as the MCP TypeScript SDK does
(`checkResourceAllowed`). For them, an identifier naming the service root, such
as `https://graphs.example.com`, covers every route, and one naming
`https://graphs.example.com/mcp` covers only this endpoint. Name the service root
when clients that accept a prefix should discover sign-in from any route.
The server answers the metadata both at `/.well-known/oauth-protected-resource`
and at the advertised path: the well-known prefix followed by the resource
identifier's path (`/mcp` above). Any other path below the prefix is 404. No
proxy rewrite is needed.
### Host and Origin
The SDK refuses a request whose `Host` is not allowed with 403. Allowed hosts
are `localhost`, `127.0.0.1`, `::1`, the OIDC resource identifier's authority,
and every `--mcp-allowed-host`. A server reached by another name, for example
behind a load balancer without OIDC, needs that name listed.
With OIDC, a browser `Origin` must be the resource identifier's origin. Without
OIDC there is no origin to allow, so a request carrying any `Origin` is refused
with 403. Native clients send no `Origin`.
## Connect a stdio agent
An agent that starts its MCP servers as child processes, such as Claude Code or
Codex in a terminal, connects to an existing server through the CLI:
```json
{"command": "omnigraph", "args": ["mcp", "--target", "staging"]}
```
`omnigraph mcp --target NAME [--graph G] [--write]` relays each JSON-RPC message
on stdin to a server or managed target's `/mcp`, one `POST` each, and writes each answer
as one line on stdout; diagnostics go to stderr. Tools, their logic and their
refusals stay the server's.
* **Credential.** Requests use the same credential acquisition as served graph
commands. Server targets use their `auth`; managed targets exchange the
pinned profile's credential for an identity bound to that cluster ([authentication providers](https://www.omnigraph.dev/docs/cli/targets.md#authentication-providers));
it never appears in the agent's configuration or output. A target whose
`auth` is `provider: anonymous` sends none. On server targets, `--as ACTOR` sends
`Omnigraph-If-Actor` with every request, so another actor's token is refused
with `412 actor_mismatch`. Managed targets refuse `--as` before any I/O.
* **One graph.** `--graph`, else the target's `default_graph`; without either the
command refuses before any request. A tool call that omits `graph` gets it, a
call or resource URI naming another graph is refused with a JSON-RPC error, and
`graphs` and `omnigraph://graphs` list only this graph. Run one bridge per
graph.
* **Writes need `--write`.** Without it, the bridge lists and admits only the
tools the server annotates `readOnlyHint: true`, and refuses a call to any
other before sending it. The applied policy still decides every call.
* **Existing endpoints only.** A server or managed target must name its existing
endpoint. Direct graph, root and catalogue targets refuse with
`target_kind_unsupported` before any I/O. The bridge opens no storage and never
starts a server or enables its MCP route.
The bridge relays up to 16 requests at once, the server's own limit, and
answers them as they complete. It applies `notifications/cancelled` itself: the
cancelled request's exchange is dropped and it gets no answer, unless its answer
was already written. A cancelled write may still have landed: read the target
before running it again. It never sends a request again:
| Failure | JSON-RPC error `data` |
| ---------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| The server refused the request over HTTP (401, 412, 429, …) | `status` and the server's error body as `error`. A 401 also carries the server's `WWW-Authenticate` challenge as `www_authenticate` when it sends one. Managed credentials reflected in error bodies or challenges are redacted |
| Nothing was sent: the server could not be reached, or speaks another HTTP contract | `error.code` `service_unavailable`, or `api_contract_mismatch`, with `effects: none` |
| The answer was lost, or took longer than 45 seconds | `outcome_unknown` with `effects: unknown` for a call to a tool that may write; otherwise `service_unavailable`, `deadline_exceeded` or `result_too_large` with `effects: none` |
These errors carry code `-32000`; refusals by the graph binding or write consent
carry `-32602`, with a `bad_request` body. After a 401 the bridge reads the
credential from its source again once, for later requests: a token rewritten
in a `file` source, or returned anew by an `exec` helper, reaches them without
a restart. An `env` source is fixed when the bridge starts, and a token
replaced with `omnigraph credentials set` belongs to a new credential
generation, so both take a restart; the error says how to replace the token. A
server that stays unreachable fails each request; the bridge keeps running
until stdin closes. Managed resource credentials renew through the selected
profile's normal cache before expiry. A refused managed credential is removed
only if it is still the cached one, and renewed for later messages; the refused
message is never replayed. If an expiring credential runs out before a message
is sent, including during discovery, the bridge renews it once before sending.
If the replacement also expires, the message fails without being sent. Renewal
never starts an interactive login.
An expiring managed profile's `exec` credential is also renewed before later
requests. The verified organization and principal must remain the same for the
life of the bridge; a changed identity requires restarting it. Organization
service permissions remain current provider facts. Managed error messages,
including JSON-RPC tool errors, never expose a reflected resource credential.
## Settings
| Flag | Environment | Default | Effect |
| ---------------------------------------------- | --------------------------------- | ------- | -------------------------- |
| `--mcp-allowed-host `, repeatable | `OMNIGRAPH_MCP_ALLOWED_HOSTS=a,b` | none | Adds allowed `Host` values |
Settings are read once at boot. `/mcp` lists every tool below, and each call
runs under the same policy action as its HTTP route, so a caller can do over
MCP exactly what its credential may do over HTTP. To restrict what an agent
may do, narrow the policy for the agent's credential.
The removed `--mcp-allow-adhoc` and `--mcp-allow-writes` flags make
`omnigraph-server` refuse to start; leftover `OMNIGRAPH_MCP_ALLOW_*` variables
are ignored, so such a deployment lists every tool. A token with no applied
policy still only reads: `query` answers, and a write tool returns 403.
## Tools
### Read tools
| Tool | Arguments | Runs | Policy action |
| --------------------- | ------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- |
| `health` | none | `GET /healthz` | none |
| `graphs` | none | `GET /graphs/discovery` for an identity credential; otherwise `GET /graphs`, narrowed to ids and names | identity, or `graph_list` |
| `schema` | `graph` | `GET /graphs/{graph_id}/schema` | `read` |
| `queries` | `graph` | `GET /graphs/{graph_id}/queries` | `read` |
| `query` | `graph`, `query` (source) or `name` (stored), optional `params`, `branch`, `snapshot` | source: `POST /graphs/{graph_id}/query`; name: `POST /graphs/{graph_id}/queries/{query_name}` as a read | source: `read`, and `change` for `explain load csv`; name: `invoke_query`, then `read` |
| `snapshot` | `graph`, optional `branch` | `GET /graphs/{graph_id}/snapshot` | `read` |
| `access` | `graph`, optional `branch`, `target_branch` | `GET /graphs/{graph_id}/access` | none |
| `branches_list` | `graph`, optional `detailed` | `GET /graphs/{graph_id}/branches` | `read`; with `detailed`, details only for branches the caller may `read` |
| `branches_diff` | `graph`, `source`, optional `target`, `kind`, `type`, `op`, `page_token`, `limit` | `GET /graphs/{graph_id}/branches/diff` | `read` on both branches |
| `branches_merge_base` | `graph`, `source`, optional `target` | `GET /graphs/{graph_id}/branches/merge-base` | `read` on both branches |
| `commits_list` | `graph`, optional `branch`, `listing`, `limit`, `page_token` | `GET /graphs/{graph_id}/commits` | `read` |
| `commits_get` | `graph`, `commit_id` | `GET /graphs/{graph_id}/commits/{commit_id}` | `read` |
| `commits_changes` | `graph`, `commit_id`, optional `kind`, `type`, `op`, `page_token`, `limit` | `GET /graphs/{graph_id}/commits/{commit_id}/changes` | `read` |
| `diff` | `graph`, `from`, `to`, optional `kind`, `type`, `op`, `page_token`, `limit` | `GET /graphs/{graph_id}/diff` | `read` on the branch each side resolves to |
| `changes_poll` | `graph`, optional `branch`, `cursor`, `start`, `kind`, `type`, `op`, `page_token`, `limit` | `GET /graphs/{graph_id}/changes` | `read` |
| `blobs_stat` | `graph`, `entity`, `type`, `id`, `property`, optional `branch`, `snapshot` | `HEAD /graphs/{graph_id}/blob` | `read` |
`graph_list` is a server-scoped action, and a server without an applied cluster
policy denies every server-scoped action. So with a static or other non-identity
token, `graphs` returns 403 until the cluster policy permits `graph_list`; the
other tools still work with a graph id the user supplies.
The server's `initialize` instructions tell a model to call `graphs`, then
`schema`, then `query` with a stored read's `name` or its own GQ source.
* `queries` lists every stored query with its typed parameters (`name`,
`kind`, `nullable`, plus `item_kind` for lists and `vector_dim` for
vectors). An entry with `mutation: true` is a stored mutation, which
`mutate` runs.
* `query` refuses stored mutations; `mutate` runs them.
* `commits_list` takes `listing`: `history` (the default), `unpublished` for
proposals written `as commit` that await `commits_publish`, or `unattached`
for snapshots written `as unattached branch`. Only `history` pages.
* `diff`, `branches_diff` and `commits_changes` page as their routes do:
continue with the returned `next_page_token` as `page_token`, keeping the
other arguments. A `diff` side the caller cannot read answers the 404 of a
side that names nothing.
* `blobs_stat` answers what `HEAD /blob` answers in its headers, as the body
`omnigraph blob stat --json` prints: the `selector`, `kind` (`managed`, with
`size` and `etag`, or `external`, with `uri`) and the resolved `target`. It
reads no payload byte. The route's conditional headers are not arguments.
* `query` with `query` takes GQ source with typed parameter declarations, the
same body as `POST /query`; `name` alongside source picks one declared query.
Source also answers `branch list`, `branch diff`,
`branch merge-base`, `branch merge … dry run` and `explain query …`.
`explain load csv …` requires `change` on its target branch before probing
source metadata; it reads no payload byte and publishes nothing. It accepts
`branch`, while refusing `snapshot`, `name` and `params`.
The tool refuses `insert`, `update`, `delete` and branch writes with the route's 400.
### Write tools
| Tool | Arguments | Runs | Policy action |
| ----------------- | -------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------- |
| `mutate` | `graph`, `query`, optional `name`, `params`, `branch`, `if_graph_commit` | `POST /graphs/{graph_id}/mutate`, or `/mutate/if-graph-commit` with `if_graph_commit` | `change`; a branch statement takes its branch action |
| `mutate`, stored | `graph`, `name`, optional `params`, `branch`, `if_graph_commit` | `POST /graphs/{graph_id}/queries/{query_name}`, or `/queries/{query_name}/if-graph-commit` with `if_graph_commit` | `invoke_query`, then `change` |
| `load` | `graph`, `mode`, `data`, optional `branch`, `from` | `POST /graphs/{graph_id}/load` | `change`, plus `branch_create` when `from` creates the branch |
| `blobs_clear` | `graph`, `entity`, `type`, `id`, `property`, optional `branch`, `if_match` | `DELETE /graphs/{graph_id}/blob` | `change` |
| `branches_create` | `graph`, `name`, optional `from`, `from_commit` | `POST /graphs/{graph_id}/branches` | `branch_create` |
| `branches_delete` | `graph`, `name` | `DELETE /graphs/{graph_id}/branches/{branch_name}` | `branch_delete` |
| `branches_merge` | `graph`, `source`, optional `target` | `POST /graphs/{graph_id}/branches/merge` | `branch_merge` |
| `commits_publish` | `graph`, `commit_id` | `POST /graphs/{graph_id}/commits/{commit_id}/publish` | `commit_publish` on the commit's recorded target |
| `commits_drop` | `graph`, `commit_id` | `DELETE /graphs/{graph_id}/commits/{commit_id}` | `commit_drop` on the commit's recorded target |
`mutate` runs a stored mutation when it is given `name` and no `query`, as
`omnigraph mutate NAME` does; `query` still refuses one. `commits_publish` and
`commits_drop` answer a commit whose control the policy denies with the 404 of
an unknown id. `commits_drop` answers an empty object, since its route answers
204 with no body. `blobs_clear` takes the route's `If-Match` value as
`if_match`; a value that does not hold fails with 412 and
`blob_precondition_failure`.
Write tools follow one contract:
* **Conditional writes.** Pass a read's `graph_commit_id` to `mutate` as
`if_graph_commit`, for an ad-hoc or a stored mutation. If the branch has
moved, the call fails with 412 and nothing is written. `if_graph_commit`
always selects the route that requires it, so an empty or malformed value is
refused with 400 and never becomes an unconditional write.
* **No retries.** The server never replays a tool call.
* **A lost response is an unknown outcome.** A write runs to completion even when
the caller stops waiting. A write still running after 30 seconds, or one whose
completion the server cannot confirm, returns `isError` with
`code: outcome_unknown`. A write the client cancelled, timed out or lost the
connection for returns nothing, and means the same. Before replaying it, read
the target with `query` or `snapshot` and compare `graph_commit_id`, or list the
branch with `commits_list`; replay a `mutate` with `if_graph_commit`, so a write
that did land fails with 412 instead of applying twice. The 30 second wait is
below the MCP TypeScript SDK's default 60 second request timeout; a client with
a shorter timeout loses the answer.
* **Size.** The whole call, including `load` data, fits in the 64 KiB request.
Use `POST /load/ndjson` for larger loads.
Schema changes and cluster deployments are not MCP tools; operators run them with
`omnigraph cluster plan` and `apply`. Byte streams are not tools either: Blob
payloads (`GET` and `PUT /blob`), `export`, and the change-feed baseline, whose
NDJSON snapshot must be installed before its cursor. Use the HTTP routes or the
CLI for them.
## Resources
| URI | Content |
| ----------------------------------------------------------------------------------------- | -------------------------------------------------- |
| `omnigraph://graphs` | The `graphs` result, JSON, under the same rule |
| `omnigraph://graphs/{graph}/schema` (template) | The graph's `.pg` source |
| `omnigraph://graphs/{graph}/branches` (template) | The `branches_list` result, JSON |
| `omnigraph://best-practices/index` | Which reference to read for which task, Markdown |
| `omnigraph://best-practices/{topic}` for `queries`, `data`, `schema`, `search`, `changes` | The agent skill's reference of that name, Markdown |
The best-practices references are the `omnigraph` agent skill's own files,
compiled into the server, so they always match its version.
A resource read runs the same handler and policy action as its tool. A refusal
is a JSON-RPC error whose `data` carries the refused tool call's content below;
an unknown resource or graph is error `-32002`.
## Results and errors
A successful call returns the route's JSON response body as structured content,
plus a text copy for older clients. Three tools answer otherwise: `graphs`
narrows `GET /graphs` for a credential that is not an identity, `blobs_stat`
answers the `HEAD /blob` headers as `blob stat` JSON, and `commits_drop`
answers an empty object for the route's 204. `health`, `graphs`, `schema`,
`queries`, `access` and `branches_list` declare an output schema.
A refused call returns `isError: true` with
`{"status": , "error": }`: the error body every HTTP
route answers, with its `code`, `disposition` and details, and the status the
same failure answers over HTTP, for example 403 for a policy denial, 404 for an
unknown or hidden graph, 503 for a graph that is not available. The MCP layer's
own refusals use the same body: `unauthorized` without a verified credential,
`too_many_requests` beyond the concurrent limit, and `cancelled`,
`deadline_exceeded` or `result_too_large`, which only MCP sends and which carry
no `status`. A write that may have happened, one whose call was cancelled or
passed its wait included, answers `outcome_unknown` with `effects: unknown`.
## Limits
| Limit | Value |
| --------------------------------------------- | --------------------------------------- |
| Request body | 64 KiB |
| Complete tool result, including the text copy | 1 MiB |
| Wait for one read | 30 seconds |
| Wait for one write | 30 seconds; the write continues past it |
| Concurrent tool executions | 16 |
A cancelled or expired call stops the caller's wait. The execution keeps its
input reservation and slot until it finishes, and there is no later result
lookup. See [admission and shutdown](https://www.omnigraph.dev/docs/getting-started/deployment.md).
## Logs
Verbose MCP SDK logs contain query arguments and results, so the server keeps
them out of its log; see [Logs](https://www.omnigraph.dev/docs/operations/server.md#logs).
# Server logs and traces
Source: https://www.omnigraph.dev/docs/operations/observability
The server writes text logs to stdout by default. Set
`OMNIGRAPH_LOG_FORMAT=json` for structured logs. HTTP completion records include
the route template, method, status and `response_headers_ms`: elapsed time until
response headers are available, not the duration of a streaming body. Unknown
routes use ``; request URLs, query strings, headers and bodies are not
included in these records.
To export traces, point the server at an OpenTelemetry collector:
```bash
OMNIGRAPH_LOG_FORMAT=json \
OTEL_EXPORTER_OTLP_ENDPOINT=http://collector:4318 \
OTEL_SERVICE_NAME=omnigraph-server \
OTEL_RESOURCE_ATTRIBUTES=deployment.environment.name=production,service.version=build-123 \
omnigraph-server --cluster /srv/omnigraph/company-brain --bind 0.0.0.0:8080
```
Export is off unless an endpoint is explicitly set. The supported transport is
OTLP HTTP/protobuf; a general endpoint gets `/v1/traces` appended.
`OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` overrides it with an exact trace URL.
Endpoints must use HTTP or HTTPS without userinfo, query parameters or fragments.
Use HTTPS outside a trusted private network. `OTEL_SDK_DISABLED=true` disables
export without disabling logs. Normal authentication configuration still applies.
Incoming W3C `traceparent` and `tracestate` connect HTTP requests to their callers.
Owned read/write spans follow the work even when the caller stops waiting;
JSON logs include their `trace_id` and `span_id`. Context is correlation only and
never supplies an actor or changes permissions. Baggage is not imported.
The SDK's `OTEL_TRACES_SAMPLER` and `OTEL_TRACES_SAMPLER_ARG` control sampling;
the default follows the parent, or samples a root request.
Only explicitly authored request and native-operation spans are exported.
`RUST_LOG` controls stdout log events (`info` by default), independently of
trace export and sampling. Setting it to `warn`, `error` or `off` does not
disable traces; warnings and errors still retain trace correlation when INFO
logs are suppressed. The `rmcp`, `lance::dataset_events` and
`lance::file_audit` targets remain restricted to warnings/errors, even with
debug overrides. These targets can include query arguments/results, full storage
URIs and deletion predicates. `omnigraph_server::init_tracing()` also enforces
the restrictions; embedders using their own subscriber must preserve them.
A failed Blob delivery, change-route request or streamed response is logged by
its error class, never by its message; see [HTTP errors](https://www.omnigraph.dev/docs/operations/troubleshooting.md#http-errors).
Collect stdout separately for logs; the server does not send
the same logs over OTLP. Resource attributes are operator-provided metadata;
do not place secrets or graph contents in them.
Export uses a bounded queue of 2,048 spans, batches of up to 256, a five-second
flush interval and a three-second HTTP timeout. A full queue drops telemetry;
collector failure never blocks a graph operation. After serving stops, a
best-effort flush starts in the background with a five-second bound; it never
delays process exit or changes native shutdown deadlines. Process exit can
lose buffered spans; telemetry is not durable operation evidence.
# Authorization and actors
Source: https://www.omnigraph.dev/docs/operations/policy
OmniGraph uses Cedar policy bundles to authorize graph, server, and cluster
configuration actions.
Policies are declared in `cluster.yaml` and installed by `cluster apply`.
Servers load applied policies at startup and activate policy changes during
live deployment, including changes to existing graphs.
## Actions
Graph-scoped actions:
| Action | Covers |
| ---------------- | ------------------------------------------------------------------------------------ |
| `read` | Queries, snapshots, branches, and commits |
| `export` | Snapshot export |
| `change` | Mutations, loads, and source metadata probes for `explain load csv` |
| `schema_apply` | Schema changes |
| `branch_create` | Branch creation |
| `branch_delete` | Branch deletion |
| `branch_merge` | Branch merge |
| `branch_claim` | Claim of a branch's [writer epoch](https://www.omnigraph.dev/docs/operations/troubleshooting.md#writer-epoch) |
| `commit_publish` | Publication of an unpublished commit onto its recorded target branch |
| `commit_drop` | Drop of an unpublished commit |
| `maintain` | Content-preserving maintenance: optimize, index builds and the full-text rebuild |
| `reclaim` | Cleanup, which permanently removes table versions and files no retained commit needs |
| `invoke_query` | Entry to a stored query |
| `admin` | Reserved graph administration |
`graph_list` is server-scoped and controls the metadata catalog at
`GET /graphs`. It does not control the minimal identity-authenticated
`GET /graphs/discovery` route: every valid cluster identity can discover all
applied graph IDs and display names, without gaining permission to read their
schema or contents. See [signed data credentials](https://www.omnigraph.dev/docs/operations/server.md#signed-data-credentials).
`config_manage` is cluster-scoped. It authorizes configuration changes through
the identity-authorized cluster API, including stored queries, graph creation,
and policy membership/binding changes on existing graphs. Grants and revocations
are authorized by the currently applied policy, never the proposed policy.
Changing an existing
graph's schema requires that graph's `schema_apply` permission;
reading its remote schema or migration preview requires `read` on `main`.
The reserved graph `admin` action does not grant cluster management.
A stored mutation requires both `invoke_query` and `change`. A stored read
requires `invoke_query` and `read`.
`maintain` and `reclaim` are separate from `change`, so an automation account
can compact a graph without the right to write data, and can hold `maintain`
without the right to delete history. `maintain` is judged on the branch the
operation maintains (`main` for optimize) and takes
`branch_scope`; `reclaim` covers every branch's history and takes no branch
scope. `change` admits no maintenance: index builds and the full-text rebuild
need `maintain` on their branch. The engine checks these actions whenever a policy is installed on the
graph handle, which a direct command on a cluster graph does (see
[direct writes](#actor-identity)).
## Bind a policy
```yaml
# cluster.yaml
policies:
graph-access:
file: graph.policy.yaml
applies_to: [knowledge]
server-access:
file: server.policy.yaml
applies_to: [cluster]
```
A bundle may target graph IDs or the `cluster` scope, but not both. Only one
bundle may bind a given scope. `cluster validate` refuses a configuration that
breaks either rule, and the server, direct writes, `access` and the `policy`
commands refuse an applied state that breaks one, so they always select the
same bundle for a graph. The target `og_catalog` binds the
[engine catalogue](https://www.omnigraph.dev/docs/queries-search/catalogue.md), which every server serves and no
`graphs:` entry declares: its `read` rule decides catalogue reads with the
resource `og_catalog` and no branch, and the bundle is applied and activated
like a graph's.
Example graph policy:
```yaml
version: 1
groups:
readers: [act-alice, act-bob]
rules:
- id: readers-can-read
allow:
actors: { group: readers }
actions: [read]
branch_scope: any
- id: readers-can-invoke
allow:
actors: { group: readers }
actions: [invoke_query]
```
A cluster-bound bundle can contain separate configuration and inventory rules:
```yaml
version: 1
groups:
owners: [principal:alice]
rules:
- id: owners-manage-config
allow:
actors: { group: owners }
actions: [config_manage]
- id: owners-list-graphs
allow:
actors: { group: owners }
actions: [graph_list]
```
`config_manage` uses the Cedar `Cluster::"root"` resource; `graph_list` uses
`Server::"root"`. Put them in separate rules and do not give either a branch
scope. Bind this file with `applies_to: [cluster]` as above.
For an identity-authorized apply, the **current applied policy** authorizes all
planned effects before any graph or configuration effect. A proposed policy
cannot give its author permission to install itself. A saved plan does not
transfer its author's permissions: another caller must pass the current policy
checks. Explicit first initialization can install a declared initial management
policy under an exact bootstrap capability; missing policy on an existing
cluster is an error, not permission to bootstrap again. Existing storage-holder
cluster APIs keep their explicit trust boundary.
Graph rules may use `branch_scope` for a source branch or
`target_branch_scope` for a destination branch. Values are `any`, `protected`,
or `unprotected`; a rule may not set both. A branch is protected when the
bundle's `protected_branches` list names it exactly:
```yaml
protected_branches: [main, release]
```
Branch names are compared as written. A `protected_branches` entry that is
empty, has leading or trailing whitespace, or contains a control character
fails policy validation, and a request naming such a branch is refused as a
bad request before any rule is evaluated (see
[branch names](https://www.omnigraph.dev/docs/branching.md#branch-names)). Server actions and graph-wide
`invoke_query` and `reclaim` rules do not take branch scopes. `commit_publish` and
`commit_drop` take `target_branch_scope`: the scope is the branch the
unpublished commit recorded as its target. `branch_claim` takes
`target_branch_scope` too: the scope is the branch the claim names. Writing an unpublished commit
needs no new action: a mutation `as commit` is a `change` on its branch and a
merge `as commit` is a `branch_merge`, so an actor with ordinary write rights
can omit `as commit`; this is a voluntary review workflow, not an approval
boundary (see [Unpublished graph commits](https://github.com/ModernRelay/omnigraph/blob/main/docs/rfcs/2026-10-10-unpublished-graph-commits.md)).
A mutation `as unattached branch` is likewise a `change` on the branch it
records as its target: the first write's branch, and for a write `on ""`
the target the base snapshot recorded, whatever branch the request names.
`commit list --unattached` and a query `--snapshot ` of an unattached
snapshot need `read` on that recorded target; `commit show ` keeps its
scope-free `read`; `commit drop` takes `commit_drop` on the recorded target as
for an unpublished commit (see [Unattached graph branches](https://github.com/ModernRelay/omnigraph/blob/main/docs/rfcs/2026-10-11-unattached-graph-branches.md)).
`branch_create` from a branch checks that branch as the source and the new
branch as the target. A start at a commit (`from_commit`) checks the branch
that wrote the commit as the source; a denial answers `404`, as an id no
commit has does, so the id discloses nothing (see
[start a branch at a commit](https://www.omnigraph.dev/docs/branching.md#start-a-branch-at-a-commit)).
## Validate and test
Policy commands read the applied bundles from a cluster and ask the bundle a
server asks: graph actions go to the bundle bound to `--graph`, and
`graph_list` and `config_manage` go to the bundle bound to `cluster`.
`validate` checks each of the two that is applied, as its own kind; `test`
runs each case against the bundle for its action. An action whose scope has
no applied bundle is refused; the other scope's bundle never answers it.
```bash
omnigraph policy validate --target ./company-brain --graph knowledge
omnigraph policy test --tests policy.tests.yaml \
--target ./company-brain --graph knowledge
omnigraph policy explain \
--target ./company-brain --graph knowledge \
--actor act-alice --action read --branch main
```
Edit the policy file or its `applies_to` bindings, then run
`cluster apply --target NAME --config DIR` using a registered server target.
The server finishes admitted requests before activating matching HTTP and
engine permissions. Later requests use the
new rules. Cluster management policy changes take effect in the same deployment;
a newly granted administrator cannot authorize that grant.
## Learn your access
`GET /graphs/{graph_id}/access` tells the caller what its own credential may do
on a graph, so a client can show only the actions it is allowed instead of
learning them from a refusal:
```bash
curl -H "Authorization: Bearer $TOKEN" -H "Omnigraph-Http-Api: 0.14" \
"https://graph.example/graphs/knowledge/access?branch=feature&target_branch=main"
```
With the example graph policy above, `act-alice` gets:
```json
{"actor_id":"act-alice","graph_id":"knowledge","graph_wide":["read","invoke_query"],
"branch":"feature","target_branch":"main","actions":["read"]}
```
* `client_id` appears when the credential is a verified OIDC access token: the
OAuth client it was issued to. The actor stays the person's
`principal:`, so policy decides alike for every client a person uses.
* `graph_wide` is the decision with no branch named: `invoke_query`, and an
action one rule allows on every branch, with no branch scope, `branch_scope:
any` or `target_branch_scope: any`. Rules that cover every branch only
together, one for protected branches and one for the rest, allow each named
branch but not the decision without one, so read `actions` for a branch.
`read` here also decides the reads that name no branch: listing branches, the
schema, `show` and `commit show`. A commit list authorizes `read` on its
branch, `main` when the query names none.
* With `branch`, `actions` lists the actions allowed on it: `read`, `export`,
`change` and `maintain` (served optimize asks it on `main`), and
`branch_delete`, `commit_publish`, `commit_drop` and `schema_apply` with that
branch as their target. Schema apply and the
system-column upgrade target `main` only, and a storage upgrade authorizes
each live branch, so `schema_apply` on another branch does not make schema
apply possible there. `branch_claim` is not listed: no command or route
performs it in this release.
* With `target_branch` as well, `actions` also lists `branch_create` from
`branch` into the target and `branch_merge` of `branch` into it.
The route evaluates the installed policy for the caller's resolved actor, each
action shaped as the route that performs it shapes it, so it answers exactly as
those routes decide. It needs no action of its own, reads no graph data, and
judges branches by name without checking that they exist. It names actions,
never the rules behind them, and reveals nothing about other actors. A
credential on a graph without a policy gets nothing (a static token gets `read`
when the server sets `OMNIGRAPH_ALLOW_READS_WITHOUT_POLICY=1`), as those routes
decide; a server without
authentication allows every graph action and reports `actor_id: null`. The MCP
`access` tool returns the same body, and `omnigraph access` prints it for a
server or managed target ([what you may do at a target](https://www.omnigraph.dev/docs/cli/targets.md#what-you-may-do-at-a-target)).
For a direct cluster target it evaluates the policy the cluster applied to the
graph, which also judges the CLI's direct writes and maintenance there. Direct
reads are not judged: storage access is their trust boundary.
It reports policy actions, not actor views, so a view can still refuse a
`change` the policy allows. It does not report `reclaim`, since no server
route runs cleanup.
## Actor identity
For HTTP requests, the server maps the bearer token to an actor. Headers,
query parameters, and request bodies cannot override that identity.
Signed credentials use `principal:`; groups and
permissions come from applied policy. An identity credential contains no
graph/action grants. Version-1 restricted credentials are not accepted.
For direct CLI writes, actor resolution is:
1. `--as `;
2. no actor.
The version-1 operator document has no actor field.
A direct write to a cluster graph, through its root target with `--graph`
or through its own store, installs the policy the cluster applied to that
graph, the policy `omnigraph access` reports for that target. Every write and
maintenance command is then judged by it before any effect, and a missing actor is
denied. A standalone graph has no policy. A direct actor is self-declared, so
this keeps a storage owner within the policy rather than authenticating them:
storage permissions remain the trust boundary. A refused direct write, like
every direct writer, keeps the cluster's admission lock
([S3 cluster roots](https://www.omnigraph.dev/docs/operations/s3-cluster-roots.md)). Only the server resolves a
served write's actor; there `--as` is an assertion the server checks against
the token ([Assert the actor](https://www.omnigraph.dev/docs/operations/server.md#assert-the-actor)).
Successful graph commits record the actor for the whole atomic change. Inspect
the audit trail with:
```bash
omnigraph commit list --target ./graph.omni --json
omnigraph commit show --target ./graph.omni --json
```
## Server startup modes
For static token authentication:
| Tokens | Policy | Startup and authorization |
| ---------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| none | none | Requires explicit `--unauthenticated`; otherwise startup fails |
| configured | none | Every action is denied, reads included; `OMNIGRAPH_ALLOW_READS_WITHOUT_POLICY=1` lets static tokens `read` ([authentication](https://www.omnigraph.dev/docs/operations/server.md#authentication)) |
| configured | configured | The policy decides each action |
| none | configured | Startup fails because no request could establish an actor |
`GET /graphs` is denied unless a `cluster`-scoped policy grants `graph_list`,
including when graph policies exist.
Signed-token trust also enables authenticated startup. Signed credentials
require an explicit policy permit for protected operations even when no
static tokens are configured. Only the minimal identity discovery route is
independent of policy membership; it exposes existence, not graph access.
Policy is enforced for graph writes inside the engine as well as at the HTTP
boundary. This keeps direct and embedded writers subject to the same action
checks when a policy engine is installed. Per-entity and per-property
authorization is not currently supported; authorization is graph/branch scoped.
# Recorded operations
Source: https://www.omnigraph.dev/docs/operations/recorded-operations
A cluster server can run `optimize` on a graph's `main` itself, so a scheduler
or an operator starts it with one HTTP request and looks the result up later
by an id it chose. The run uses the same engine as
[`omnigraph optimize`](https://www.omnigraph.dev/docs/operations/maintenance.md#optimize) and publishes one graph commit.
Every request and every outcome is kept in the cluster's durable operation
record, so a caller that lost a response learns what happened instead of
guessing. `optimize` is the first recorded kind; lookup and cancel serve every
kind.
The routes are off by default. A standalone server, which has no cluster
record, refuses them; direct `omnigraph optimize` stays available there.
## Enable it
| Variable | Default | Meaning |
| ------------------------------------------------------------------------------------------------------------------------ | ----------- | ---------------------------------------------------------------------------------------------------------------- |
| `OMNIGRAPH_OPERATION_ROUTES` | `false` | `true` serves the routes; otherwise they answer 503. |
| `OMNIGRAPH_OPERATION_RESOLUTION_WALK_LIMIT` | `256` | Commits one lookup or submission reads of a branch's history to settle a publication in doubt. |
| `OMNIGRAPH_OPTIMIZE_GATE_HOLD_SECONDS` | `60` | The gate hold of a request that names none. |
| `OMNIGRAPH_OPTIMIZE_MAX_GATE_HOLD_SECONDS` | `300` | The largest gate hold a request may ask for. |
| `OMNIGRAPH_OPTIMIZE_ARTIFACT_WRITE_LIMIT_BYTES` | `536870912` | The largest adjacency artifact a served run writes. |
| `OMNIGRAPH_OPTIMIZE_PLANNING_CONCURRENCY` | `8` | Tables a served run plans ahead of the one it rewrites; it rewrites one at a time. |
| `OMNIGRAPH_OPTIMIZE_MAX_SOURCE_FRAGMENTS`, `_MAX_SOURCE_ROWS`, `_MAX_SOURCE_BYTES`, `_MAX_BLOB_BYTES`, `_MAX_INDEX_ROWS` | unset | The per-table ceilings every served run applies ([bounding a run](https://www.omnigraph.dev/docs/operations/maintenance.md#bounding-a-run)). |
| `OMNIGRAPH_OPTIMIZE_RUN_SOURCE_BYTES` | unset | The run byte budget. |
A value that does not parse, a default above its ceiling, or a table byte
ceiling above the run budget stops the server at startup. The ceilings are the
server's: a request names only its tables and how long writers wait. Every
route needs the [`maintain`](https://www.omnigraph.dev/docs/operations/policy.md#actions) action on the graph's `main`
branch, for the actor the bearer token resolves to.
## Submit, look up, cancel
```bash
curl -X POST "$SERVER/graphs/knowledge/operations/optimize" \
-H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
-d '{"operation_id": "01JC0000000000000000000000", "max_gate_hold_ms": 30000}'
curl "$SERVER/graphs/knowledge/operations/01JC0000000000000000000000" \
-H "Authorization: Bearer $TOKEN"
curl -X POST "$SERVER/graphs/knowledge/operations/01JC0000000000000000000000/cancel" \
-H "Authorization: Bearer $TOKEN"
```
`operation_id` is a canonical ULID the caller generates. The body also takes
`tables` (run only those table keys, in that order) and `max_gate_hold_ms`,
at or below the server's ceiling; nothing else.
A submission answers 202 with the record once it is durably accepted, and 200
once the operation is terminal. Submitting the same id with the same request
answers the existing record; it never starts the work again. A lookup answers
200 with the record, or `"lookup": "unknown"` when the record does not hold
the id. `unknown` does not prove the operation never ran, and it is never a
reason to resubmit the same work under the same id.
| Refusal | Status |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- |
| The same id with a different request (message prefix `[operation_request_mismatch]`); an id at or before the record's eviction mark (`[operation_id_expired]`, mint a new one); an outstanding deployment; four unresolved publications; a record from another ledger at this root (`[operation_record_foreign]`); a cluster state file whose text is not valid cluster state (`[invalid_state_json]`) or an operation record whose text is not a valid record (`[operation_record_invalid]`; restore the object from a trusted copy, while bytes that are not text answer 500) | 409 `conflict` |
| Another operation is active in the maintenance scope (its id is in the message), or the scope is admitting one | 429 `too_many_requests`, with `Retry-After` |
| A gate hold above the server's ceiling, an `operation_id` that is not a canonical ULID, or one dated more than a minute ahead of the server's clock | 400 `bad_request` |
| The routes are off, the server is draining, the graph is closing for a deployment, a storage read of the cluster state file, its lock or the operation record failed temporarily or for an unknown reason (message prefix `[state_read_error]`, `[state_lock_read_error]` or `[operation_record_read_error]`; send the same request again), or the record write was refused and nothing was accepted (`effects: none`). A refusal that names an earlier request of unknown outcome clears once a read sees the record change; for an unconditional write, or a create whose record never appeared, it holds until the server restarts | 503 `service_unavailable` |
| A record write without a definite answer | 500 `outcome_unknown`; look the id up |
| A storage read of the cluster state file, its lock or the operation record failed in a way that repeats on every read: bytes that are not text, an object over its size bound, or access the store denies (same message prefixes; `effects: none`, nothing changed). Sending the same request again fails the same way until the object or the storage configuration is repaired | 500 `internal` |
## From the CLI
`omnigraph optimize --target ` runs the same route. It prints the id
it minted before it sends, waits for the terminal record, and after a lost
response looks the id up instead of submitting again. `--no-wait` returns
once the run is accepted, `--operation ` observes an existing operation,
and `--timeout` bounds the wait, not the operation. `--tables` and
`--max-gate-hold` map to the request.
| Outcome | Exit |
| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---- |
| `succeeded`, or accepted or observed with `--no-wait` | 0 |
| Refused before acceptance, or `failed`, `cancelled` or `interrupted` | 1 |
| Still `publishing` or running when the wait ended, `unsettled`, `unknown` after a lost response, a lookup refused after acceptance, or five lookups in a row without a definite answer | 2 |
| The same id with a different request | 4 |
| Another operation is active in the scope | 75 |
## States
An operation moves through `accepted`, `running` and `publishing` to one
terminal state:
* `succeeded`: it published a commit (`published_commit`), or found nothing to
publish. A succeeded run may still have deferred or failed tables; read each
table's outcome under `outcome`.
* `failed`, `cancelled` and `interrupted`: nothing was published, and the
staged work is reclaimed by a later `cleanup`.
* `unsettled`: the graph was deleted while the publication was in doubt
(`reason: graph_deleted`), so whether it published is unknown and nothing
can answer it any more.
`publishing` means a commit may have landed. When the server cannot tell, the
operation stays `publishing` and leaves the active slot. Once that graph's
`main` has a newer commit, the next submission or lookup settles it from
`main`'s history: `succeeded` if the run's commit is there, otherwise
`failed`. Each attempt reads one page of that history; a record whose walk is
still open shows `resolution` with the commit the next page starts at and how
many commits it has covered. An operation that was `accepted` or `running`
when its server stopped is `interrupted` at the next submission or lookup.
A full-catalog run that a gate hold or the byte budget stopped records the
table to start at; the next full-catalog submission for that graph starts
there, so successive bounded runs reach every table.
Cancellation stops the run at its next safe boundary. A run that already
started its publication finishes it and reports that the cancellation arrived
late. A shutdown and a deployment that affects the graph cancel the running
operation the same way, and the record names the reason.
The record keeps the 32 newest terminal operations and the time of the newest
id it evicted: an older id answers `unknown`, and a submission under an id
minted at or before that time is refused as expired, so an evicted operation
never runs again. A retained id always answers its record first, whatever the
mark says. Because the mark is a caller's time, an id dated more than a minute
ahead of the server's clock is refused before it is accepted; otherwise one
such id, once evicted, would refuse every honest id after it.
# S3 cluster roots
Source: https://www.omnigraph.dev/docs/operations/s3-cluster-roots
A cluster whose server is stopped has no served route for deployment, recovery
or maintenance. For that case `cluster plan`, `cluster apply`, `cluster status`,
`cluster force-unlock`, `cluster upgrade-ledger` and the four
[maintenance commands](https://www.omnigraph.dev/docs/operations/maintenance.md) (`optimize`, `index build`, `index rebuild`
and `cleanup`, and the deprecated `rebuild-full-text-indexes`) accept an
inline `s3://` cluster root. Maintenance selects its graph with `--graph`; a graph
URI is refused. Every other command, and any `az://`
address, refuses an inline cloud target.
```bash
omnigraph --target s3://bucket/company-brain cluster status --json
omnigraph optimize --target s3://bucket/company-brain --graph knowledge --as act-operator
```
A direct write or maintenance command on a graph with an applied policy is
judged by it, so `--as` names an actor the policy grants the command
(`maintain` for `optimize`, `reclaim` for `cleanup`); see
[direct writes](https://www.omnigraph.dev/docs/operations/policy.md#actor-identity).
Every direct writer on a root takes the cluster's admission lock and keeps it
after it finishes, as a direct `cluster apply` does. A stopped server's lock
blocks the first direct command, and each command's lock blocks the next one
and the server's boot. Read the held `lock_id` from `cluster status --json`,
establish that its owner has settled, and release that exact ID with
`cluster force-unlock` before the next writer or the server starts.
The destination is captured once from the environment: the region from
`AWS_REGION` or `AWS_DEFAULT_REGION` (required), the endpoint from
`AWS_ENDPOINT_URL_S3`, then `AWS_ENDPOINT_URL` or `AWS_ENDPOINT`, and the
addressing style from `AWS_S3_FORCE_PATH_STYLE` or
`AWS_VIRTUAL_HOSTED_STYLE_REQUEST`. With `AWS_PROFILE` set the endpoint must be
explicit. Unprefixed aliases such as `REGION` or `ENDPOINT`, conflicting
endpoint aliases and S3 Express refuse. Other environment settings supply
authentication only. The stderr target line reports the captured provider
before any I/O. `cleanup --confirm` against an S3 root also needs `--yes`, as
for any non-local target.
# HTTP server
Source: https://www.omnigraph.dev/docs/operations/server
`omnigraph-server` serves every healthy graph in one applied cluster under
`/graphs/{graph_id}/…`. It has no single-graph boot mode. Directory boot reads
the current `cluster.yaml` to resolve storage and validate the source location;
graph, query, and policy resources come from applied state. URI boot is
config-free.
The [OpenAPI document](https://github.com/ModernRelay/omnigraph/blob/main/openapi.json) defines the graph API. A running
server returns it from `GET /openapi.json`, which is not self-listed.
## Start a server
From a local cluster bundle:
```bash
OMNIGRAPH_SERVER_BEARER_TOKENS_JSON='{"act-alice":"secret"}' \
omnigraph-server --cluster ./company-brain --bind 0.0.0.0:8080
```
For object storage, replace `./company-brain` with the cluster root, for example
`s3://company-data/omnigraph/company-brain`.
The default bind address is `127.0.0.1:8080`. `--require-all-graphs` makes any
graph startup failure fatal. Otherwise failed graphs remain in the authorized
inventory as blocked while healthy graphs serve; see [readiness](https://www.omnigraph.dev/docs/getting-started/deployment.md).
An applied empty cluster creates no default graph and serves an empty inventory.
`/readyz` reports its applied digest, ledger revision/CAS and zero graph counts.
Missing or unapplied state, or a nonempty cluster whose graphs all fail, refuses
startup. Authentication, policy and data-token root checks still apply.
Use `cluster apply --target NAME --config DIR` with a registered server target
for schema, query, policy, provider and Blob-rule changes without restart. Graph additions and removals
use the same deployment; removal deletes the graph's managed storage and history. Direct apply requires stopped serving and a subsequent
start; see [cluster deployments](https://www.omnigraph.dev/docs/clusters.md).
An unapplied resource edit does not activate it, although changing or breaking
the directory's config can change where boot looks for applied state.
For first boot of a fresh, empty S3 cluster initialized by the library, use
`--bootstrap-handoff FILE` with its exact receipt. This is a one-time ownership
claim, not a restart or recovery path; see
[first boot from a bootstrap receipt](https://www.omnigraph.dev/docs/clusters.md#first-boot-from-a-bootstrap-receipt).
### Logs
Use [server logs and traces](https://www.omnigraph.dev/docs/operations/observability.md) for JSON logging, optional OTLP
export and correlation with incoming W3C trace context.
## HTTP contract
Refresh the CLI, server and HTTP integrations together ([0.14 release notes](https://github.com/ModernRelay/omnigraph/blob/main/docs/releases/v0.14.0.md)). Every protected
request requires exactly one `Omnigraph-Http-Api: 0.14`: graph, registry and
cluster-deployment calls, including `/graphs/discovery`, JSON, streams and Blob GET/HEAD. Missing,
duplicate, combined or unsupported values return `400 api_contract_mismatch`
after authentication but before graph lookup, body decoding or execution. This HTTP identifier is
separate from package/storage versions and grants no permission. Earlier prerelease builds also use `0.14`,
so header admission cannot detect their older shapes; refresh components together and regenerate SDKs.
Ordinary responses, including errors and streams, carry the same header. Check
it before decoding or emitting the body. A missing/incompatible response header
means unknown effects, never permission to replay. Proxies must preserve it both ways.
Server URLs name the service root, including its proxy prefix. Select a graph
with `--graph`, `default_graph` or `alias.graph`. Migrate graph-qualified server
URLs to that root plus graph selection; the CLI never guesses a root from a path.
Register the endpoint under `targets:` and select its name with `--target`;
inline remote URLs and Unix socket targets are unsupported. Other clients can
use a [Unix-socket gateway](https://www.omnigraph.dev/docs/getting-started/deployment.md#unix-socket-access).
Server targets bind a credential owner or anonymous access to an endpoint scope.
Every composed route must retain that captured choice. A more specific target
with another owner or anonymous access blocks the request before discovery;
the CLI never switches credentials to match the nested route.
Public `/healthz`, `/readyz` and `/openapi.json` need no header and identify the
contract in their responses. Before every graph, registry or deployment request, the CLI sends
`HEAD /healthz` to that base with a five-second bound, no bearer, body consumption
or cache. Failure prevents data dispatch. Each request adds one discovery round
trip; its existing deadline stays separate. Neither request follows redirects
or retries automatically. Older clients/servers without this contract are unsupported.
MCP/OAuth metadata neither require nor return this header; managed control-plane
APIs keep their separate protocol.
## Authentication
Choose one static token source:
| Environment variable | Value |
| ------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `OMNIGRAPH_SERVER_BEARER_TOKEN` | One bearer token; actor is `default` |
| `OMNIGRAPH_SERVER_BEARER_TOKENS_JSON` | Actor-to-token object, e.g. `{"act-alice":"secret-a"}` |
| `OMNIGRAPH_SERVER_BEARER_TOKENS_FILE` | Path to that JSON object. On Unix a file other accounts can write is refused at startup, and one they can read logs a warning; use `chmod 600`, or `defaultMode: 0400` on a Kubernetes secret volume |
| `OMNIGRAPH_SERVER_BEARER_TOKENS_AWS_SECRET` | Secrets Manager mapping; requires an AWS-enabled build: the container images, the release archives from v0.14.0 and every edge build, or a source build with `cargo build -p omnigraph-server --features aws` |
Every source refuses a blank actor, a duplicate token value and an actor name
starting with `principal:`, which is reserved for signed and OIDC identities
(`principal:`), so a static token cannot match their policy rules.
Send `Authorization: Bearer secret-a` (the scheme is case-insensitive). The token selects the
actor for authorization and commit attribution; clients cannot claim another actor. See
[Authorization and actors](https://www.omnigraph.dev/docs/operations/policy.md). Every 401 carries a `WWW-Authenticate: Bearer` challenge, with `error="invalid_token"` when
the server refused the token sent; a valid OIDC token it does not admit gets 403 ([MCP](https://www.omnigraph.dev/docs/operations/mcp.md)).
A server with neither static tokens, signed-token trust, nor policy refuses to start unless you explicitly
pass `--unauthenticated` (or set `OMNIGRAPH_UNAUTHENTICATED=1`). Use that only
on a trusted development network. A graph without an applied policy refuses
every action to every token, reads included: bearer tokens are not scoped to
graphs, so a permit there would reach every graph that lacks a policy. Apply a
[graph policy](https://www.omnigraph.dev/docs/operations/policy.md) to each graph a token should reach. Setting
`OMNIGRAPH_ALLOW_READS_WITHOUT_POLICY=1` in the server's environment restores
the earlier behaviour, in which every static token may `read` every graph
without a policy; the server logs a warning at startup when it is set.
### Assert the actor
A client that means to act as a particular actor sends `Omnigraph-If-Actor: ` with any
protected request, `/mcp` included. After authentication and the contract check (on `/mcp`,
before admission), and before graph lookup or any effect, a match changes nothing and a mismatch
is `412 actor_mismatch` with nothing run; a server without authentication resolves no actor. A
repeated, padded or empty header is `400 bad_request`. The header never supplies the actor, and a
server with an incompatible HTTP contract refuses that contract first. The CLI sends it for `--as` on served
writes, served `cluster plan` and `apply`, and every [`mcp`](https://www.omnigraph.dev/docs/operations/mcp.md#connect-a-stdio-agent) request.
### Signed data credentials
To accept short-lived credentials from an issuer, mount its public trust file
and select it explicitly:
```bash
omnigraph-server --cluster s3://company-data/company-brain \
--data-token-trust /run/omnigraph/data-token-trust.json
```
The file binds public signing keys to the exact storage root, issuer, account,
cluster id, and cluster incarnation. Invalid trust or a root mismatch refuses
startup before graphs open. Trust alone requires bearer authentication. The
server verifies tokens locally; it does not contact an identity or control
service. The provisioning operator owns supplying the correct identity binding.
The [trust and credential format](https://github.com/ModernRelay/omnigraph/blob/main/docs/rfcs/0053-offline-data-token-verification.md#public-trust-and-root-binding)
defines the machine-written file.
Signed credentials use the actor `principal:`. A caller
cannot change its actor through request headers or JSON. The server accepts
version-2 identity credentials, which bind the principal to the cluster and
contain no permissions. Applied Cedar policy decides graph operations;
missing policy or an unknown policy actor denies protected access.
Every valid identity credential can call `GET /graphs/discovery` for applied
graph IDs and names (currently identical), including blocked graphs. It
requires no policy membership and returns no locations, availability, schema,
queries or data. Static credentials cannot use it. `GET /graphs`
requires `graph_list` permission. Discovery grants no graph access or reachability.
See [managed data access](https://www.omnigraph.dev/docs/cli/managed-data.md) for issuance and CLI discovery.
Tokens live for 60–86,400 seconds from issuance. The server permits an issuance
clock up to 30 seconds ahead, so at most 86,430 seconds can remain on admission.
Expiry has no grace period. Logout or a permission change at the issuer does
not revoke an issued token; already accepted operations can finish after
expiry. Stored-query calls need `invoke_query` plus `read` or `change` for the
body. Apply policy changes through `cluster apply --target`; editing a source
file alone does not change permissions. Deployment uses
[current-policy authorization](https://www.omnigraph.dev/docs/operations/policy.md#actions); the identity credential supplies no bypass.
Static credentials can coexist for operator recovery. An exact configured
static credential keeps its existing authority, including credentials with
dots; an invalid signed credential never falls back to static or anonymous
access. Restart to change public trust. Install new and old keys together
before issuing with a new key, and retain the old key for at least 86,430
seconds after its final issuance before removing it with another restart.
### OIDC resource identities and MCP
Enable OIDC with a public admission file alongside signed or static credentials:
```bash
omnigraph-server --cluster s3://company-data/company-brain \
--oidc-identity-trust /run/omnigraph/provider-access.json
```
The file binds an exact HTTPS issuer, resource audience, organization, account,
cluster incarnation and canonical root. Public RSA keys verify RS256 tokens;
explicit subject mappings select stable `principal:` actors. Tokens carry
no graph permissions: applied Cedar governs graph and schema operations, while
every admitted identity can discover graph IDs and names.
Human access tokens must name exactly one configured resource audience, the
configured organization and an admitted subject, and expire within 300 seconds
of issuance. OAuth-client ID tokens, delegated or impersonated credentials and
unqualified machine identities refuse. Every refresh must request the exact
resource again and check the returned audience.
Supply at most four RSA keys, 1,000 subject mappings and 256 KiB per public file.
The [resource identity contract](https://github.com/ModernRelay/omnigraph/blob/main/docs/rfcs/2026-09-09-identity-credentials-and-applied-policy.md#provider-native-access-and-standard-clients)
defines the versioned format. Authority is held by whoever can publish this
file; it contains no provider secret or graph policy. Protect its filesystem
permissions and publish complete updates atomically.
Boot validates identity and root before opening graphs. Local refresh runs every
five seconds: the binding stays fixed, revisions increase, and equal revisions
require identical bytes. Every request checks the original snapshot deadline,
at most 300 seconds after capture; invalid updates cannot extend it. Requests
never fetch keys or call a control service, so publisher outages eventually
prevent new OIDC access; accepted operations can finish. Admission refresh does
not restart writers. Graph configuration retains its normal activation.
With this profile configured, the server also serves public protected-resource metadata and names it
in every 401's challenge, so standard OAuth clients such as Claude can sign in. The [MCP guide](https://www.omnigraph.dev/docs/operations/mcp.md)
describes sign-in, which resource identifier to choose, and the endpoint's tools, settings and limits.
## Route families
| Route family | Purpose |
| ------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `GET /healthz` | Process health |
| `GET /readyz` | Replica readiness and booted applied revision |
| `GET /openapi.json` | Runtime copy of the OpenAPI document |
| `GET /graphs` | Graph metadata catalog; requires `graph_list` policy |
| `GET /graphs/discovery` | Graph IDs and display names only; requires an identity credential |
| `POST /cluster/plan` | Preview source changes against the running cluster without applying them |
| `POST /cluster/deployments` | Submit an exact-ID deployment: graph creation/deletion or schema, query, policy, provider and Blob-binding changes |
| `GET /cluster/deployments`, `GET /cluster/deployments/{deployment_id}` | Authorized deployment status and current-process activation observation |
| `GET /.well-known/oauth-protected-resource`, also at the resource's path suffix | Public OIDC resource metadata; only when OIDC trust is configured |
| `/mcp` | The [MCP endpoint](https://www.omnigraph.dev/docs/operations/mcp.md); when static tokens, signed data tokens or OIDC trust are configured |
| `/graphs/{graph_id}/query`, `/mutate` | Run inline GQ source |
| `/graphs/{graph_id}/mutate/if-graph-commit` | Run an inline conditional mutation |
| `/graphs/{graph_id}/queries` | List and invoke stored queries, including conditional mutations |
| `/graphs/{graph_id}/load`, `/load/ndjson` | Bounded batch loading |
| `/graphs/{graph_id}/blob` | GET/HEAD one Blob cell; PUT/DELETE to replace or clear it |
| `/graphs/{graph_id}/branches` | Branch management and merge, a detailed listing (`GET /branches?detailed=true`), a merge dry run (`dry_run: true`), a branch's diff against its merge base (`GET /branches/diff`) and that merge base (`GET /branches/merge-base`) |
| `/graphs/{graph_id}/snapshot`, `/commits` | Snapshot (`internal_schema_version`, `graph_manifest_version`, and the datasets of one captured graph version), history (paged with `limit` and `page_token`), per-commit changes, and the unpublished commits of a branch (`GET /commits?listing=unpublished`, `POST /commits/{commit_id}/publish`, `DELETE /commits/{commit_id}`) |
| `/graphs/{graph_id}/changes`, `/diff` | Poll a branch feed, establish a baseline, or diff two points (branches or commit ids) |
| `/graphs/{graph_id}/schema` | Show the accepted schema |
| `GET /graphs/{graph_id}/access` | The caller's own actor and the actions its credential allows; see [Learn your access](https://www.omnigraph.dev/docs/operations/policy.md#learn-your-access) |
| `/graphs/{graph_id}/export` | Stream a branch snapshot as JSONL |
`/query` also serves `branch list`, the branch reads (`branch diff`, `branch merge-base`, `branch merge … dry run`), `show`, and [explain](https://www.omnigraph.dev/docs/queries-search/explain.md).
`/mutate` serves `branch create`, `branch delete`, `branch merge`, `commit publish`,
and `commit drop`. See [Branching](https://www.omnigraph.dev/docs/branching.md).
Each of `/query`, `/mutate`, `/mutate/if-graph-commit` and `/branches/merge`
takes an optional `settings` field, and the two GET change routes a `set=`
parameter; see [Session settings](https://www.omnigraph.dev/docs/queries-search/queries.md#session-settings).
## Run an inline query
```bash
curl -sS http://localhost:8080/graphs/knowledge/query \
-H 'Omnigraph-Http-Api: 0.14' \
-H 'authorization: Bearer secret-a' \
-H 'content-type: application/json' \
-d '{
"query":"query find($name: String) { match { $p: Person { name: $name } } return { $p.name } }",
"name":"find",
"params":{"name":"Ada"}
}'
```
Use `branch` or `snapshot` to select a read view; they are mutually exclusive.
When the read snapshot has an effective graph head, the canonical `/query`
response includes its `graph_commit_id`, pinned with the returned rows. Inline
writes go to `/mutate` and may select a target `branch`.
## Invoke a stored query
```bash
curl -sS http://localhost:8080/graphs/knowledge/queries/find_person \
-H 'Omnigraph-Http-Api: 0.14' \
-H 'authorization: Bearer secret-a' \
-H 'content-type: application/json' \
-d '{"params":{"name":"Ada"}}'
```
Authorization denials for stored-query invocation appear as `404`, preventing
callers from probing registry names. A stored query then receives the normal
`read` or `change` authorization check for its body.
## Conditional mutations
Use a dedicated route and the commit returned by the read whose result you are
acting on:
```bash
curl -sS http://localhost:8080/graphs/knowledge/mutate/if-graph-commit \
-H 'Omnigraph-Http-Api: 0.14' \
-H 'authorization: Bearer secret-a' \
-H 'content-type: application/json' \
-H 'Omnigraph-If-Graph-Commit: ' \
-d '{"query":"query rename($name: String) { update Person set { name: $name } where email = \"ada@example.com\" }","name":"rename","params":{"name":"Ada"}}'
```
Stored mutations use `POST /graphs/{graph_id}/queries/{query_name}/if-graph-commit` with
that header. Ordinary `/mutate` and `/queries/{query_name}`
reject it rather than ignore the condition. Never fall back to an unconditional
route after a conditional request fails.
The header must contain one raw commit id; wildcard, quoted, weak-ETag, and
comma-list forms are invalid. The condition covers the whole target branch.
If its effective head differs, the server returns `412` with
`precondition_failure { expected, actual? }` and writes nothing. Re-read and
decide again.
Successful mutation and load responses contain `commit`, the exact commit
receipt for that attempt. A successful mutation with no matching entities
returns `"commit": null`.
## Load NDJSON
`POST /graphs/{graph_id}/load/ndjson` accepts logical node and edge records with
`Content-Type: application/x-ndjson`. The request is one bounded atomic graph
batch; it is not a durable stream or an unbounded ingestion session. Split a
larger feed into batches and wait for each response before acknowledging it
upstream.
HTTP loading defaults to `mode=merge` and branch `main`. A missing target branch
is an error unless the request supplies `from`. This differs from the CLI,
where `--mode` is always required.
See [Mutations and loading](https://www.omnigraph.dev/docs/queries-search/mutations.md) for the record shape and load
modes, including [CSV statements on `/mutate`](https://www.omnigraph.dev/docs/queries-search/mutations.md#csv-loading).
## Deliver Blob values
`GET`/`HEAD /graphs/{graph_id}/blob` select a cell by `entity`, `type`, `id` and `property`,
support managed ranges/ETag conditions, and report external references without fetching
them. `PUT` and `DELETE` replace or clear the cell; see [Blob values](https://www.omnigraph.dev/docs/schema/blobs.md),
[Replacing one Blob value](https://www.omnigraph.dev/docs/schema/blobs.md#replacing-one-blob-value) and [Blob limits](https://www.omnigraph.dev/docs/schema/blobs.md#limits).
## Changes and baselines
`GET /graphs/{graph_id}/commits/{commit_id}/changes` compares a commit with its first
parent, `GET /graphs/{graph_id}/diff?from=&to=` any two points, and `/branches/diff`
a branch with its merge base. `GET /graphs/{graph_id}/changes` polls branch commits
and returns a durable cursor; unreadable history returns `410 change_feed_gap`.
`POST /graphs/{graph_id}/changes/baseline` streams entities, a snapshot commit and a
resume cursor ([Changes, Diffs and Change Feeds](https://www.omnigraph.dev/docs/branching/changes.md)).
## Errors and retries
Every error body, an unknown route's and a malformed request's included, carries
a `code`, its details and a `disposition` (`effects`, `action`; see
[failed writes](https://www.omnigraph.dev/docs/operations/troubleshooting.md#failed-data-write-commands)); a write the
server could not settle is `outcome_unknown`, with `500`. Admission limits use `429`, size limits `413`, blocked graphs or closed admission `503`, each refusal
that asks for a retry with `Retry-After`. Admitted writes continue after disconnect.
For TLS, secrets, storage credentials, container examples and Azure admission,
see [Deployment](https://www.omnigraph.dev/docs/getting-started/deployment.md).
# Storage failures and restart
Source: https://www.omnigraph.dev/docs/operations/storage-failures
A served write sends many storage requests: data files, a transaction file,
and last the conditional create of the graph's next manifest version, which
publishes the commit. This page describes what a
client and an operator see when one of those requests fails or gets no answer.
A request's outcome belongs to the write that sent it: one failed or unanswered
request never stops the server. Process shutdown and admission are described
under [Deployment](https://www.omnigraph.dev/docs/getting-started/deployment.md#binary).
## Publication outcomes
A graph commit becomes visible only when its manifest version is created,
put-if-absent. A failure before that request is definite: nothing was
published, the write answers its error, and the server keeps serving once the
write's own storage requests have settled. That includes a failed put of the
transaction file Lance writes just before the manifest, for example one the
store throttled with `SlowDown`, and a manifest request the store refused
outright.
A manifest request whose answer was lost, timed out or throttled is re-issued
unchanged, up to four times with about three seconds of pauses in total, plus
the time spent on each storage request. A re-issue that creates the version
publishes the commit once. A version that already exists is read
back: it holds either this commit, and the write succeeds, or another commit,
and the write meets an ordinary conflict. Re-issuing the same conditional
request never publishes a duplicate. When a re-issue meets a version already
held and the read-back then finds it missing, the outcome stays unknown rather
than failed: on S3 that conflict can answer the first request, still landing.
## Unknown publications
The outcome stays unknown if no re-issue settles the request or the committed
version cannot be read back reliably. A local filesystem error can also occur
after a write has taken effect, so an error alone does not prove that nothing
was published. When the outcome remains unknown, the write answers 500
`outcome_unknown` on HTTP and MCP alike, and the server keeps serving. Each
manifest version is created only if absent, so the request lands before the
branch's next publication or never: the branch's next write either finds it and
builds on it, or takes that version first, after which the late request can no
longer land. A read that does not show the write is therefore not proof that it
failed until another commit follows its base; read the branch's commits before
repeating it.
The same rule applies when creating a proposal or an unattached snapshot: if
its final authority check fails, deleting its tag does not prove that an earlier
create request cannot still complete. That outcome remains unknown unless the
candidate is proven expired.
## Retries
The server's storage clients retry reads, and writes of fresh uniquely named
files, within Lance's own bounds: three client retries and three throttle
retries. A conditional request, such as a manifest create, and a write to a
reused key make one attempt each, and their owner resolves an ambiguous answer
itself. `OBJECT_STORE_CLIENT_MAX_RETRIES`, if set, must be `0`.
On S3 and Azure, a conditional write of a control object (the cluster lock and
state, a deployment resource, a lease heartbeat or claim) whose answer was lost,
timed out or throttled is re-issued unchanged, as a manifest request is: up to
four times with about three seconds of pauses in total. A request the store
refused before applying it, such as a missing bucket or container, fails at
once, and so does a permission or authentication error on the server. When an
earlier copy of the request may be what the object holds, the object is read
back, and holding this request's bytes makes it this write's. An object still
absent, or still at the expected version, once no re-issue settles the request
leaves the write failed with its outcome unknown.
A command that writes storage directly, without a server, uses a client that
retries a request itself before any re-issue: up to its `max_retries` (10 by
default), and on S3 an if-match that meets a conflicting write in flight too.
Those attempts are not logged. Because one of them may have applied, that
client's permission or authentication error is re-issued like a lost answer,
and its first conflict is read back.
A write to a reused key whose outcome is unknown refuses later writes to that
key only, until a clean shutdown; a clean shutdown with such a key retains
admission. Its owner's re-issue of a conditional write is still admitted. The
key settles when that re-issue applies, or when a read of the key shows its
condition no longer holds. A request the store refused before applying it
leaves its key settled.
## When the server stops
A panic, a lost write owner, a publication adopted in storage but not in
memory (`RecoveryRequired`), an opaque execution failure, or an uncertain
startup or deployment activation still closes admission for every graph and
exits 2. Their effects are not scoped to one write's storage keys.
## Restart after a failure
On S3 roots a server's cluster lock carries a lease, so an orchestrator can
restart a failed server with no operator step. Set its restart policy to
restart on failure (Railway `ON_FAILURE`, Kubernetes `restartPolicy: Always`).
* The lock's owner renews the lease with heartbeat objects under
`__cluster/lease/` every third of a lease. Heartbeats retry on their own, and
one whose answer was lost counts once a retry finds it. When no heartbeat
is answered for two thirds of a lease, the owner stops every storage write,
even after a pause of its whole process, and exits 2 without releasing. It stops the same way at its next
heartbeat, at most a third of a lease later, when a successor has claimed
the lock or `lock.json` no longer names it (after a takeover or a
`cluster force-unlock` of its ID). A server whose first heartbeat is not
answered within that time does not start serving.
* A restarted server that finds the lock held watches those heartbeats. It
refuses with `state_lock_held` while they continue, and takes the lock over
once they have been silent for a lease and a half. A takeover leaves the
previous owner's lease directory, its claim and last heartbeats, in place.
* The lease is 30 seconds by default, so a restart after a crash waits about
45 seconds. `OMNIGRAPH_CLUSTER_LEASE_SECONDS` sets it between 2 and 600
seconds; a successor waits on the value the owner recorded.
* Only a server's lock has a lease, and only a server takes a lock over. A
direct CLI write or deployment holds an unleased lock and refuses a held
lock at once; a lock one of them retains needs the exact-ID unlock.
* A request the old owner sent before it stopped can still reach storage
after the takeover. It cannot corrupt the graph: it writes a fresh file
nothing references, makes a conditional publication the new owner also
contends for, or writes a reused key whose late landing is harmless.
A lock with no lease still needs the exact-ID unlock described under
[writer topology](https://www.omnigraph.dev/docs/getting-started/deployment.md#writer-topology). Local and Azure locks have
no lease, and neither does a CLI writer's lock or a lock an older binary
wrote. A local root has no
conditional update, so a successor could not replace the lock without racing a
release the old owner already sent. Older binaries refuse the leased
(version 3) lock record; do not delete it to bypass the version check.
## Logs
No storage-failure log carries an error's message, which can name storage
paths; each records the failure's class.
* `engine failure without definite completion evidence`: the failure's class,
the status its response sends, and whether its uncertainty is scoped to its
storage keys. When the failure is a publication whose outcome is unknown, its
`publication` field names the graph commit, so you can look the commit up
later and see whether it landed.
* `owned write ended without a definite outcome`: the status its response
sends, 500 for `outcome_unknown`, and whether it contained the process.
* `manifest request outcome unknown; re-issuing the identical conditional
request`, for each re-issue,
`manifest request outcome unknown after every re-issue` when none settles,
and `manifest version held after a re-issue` when a re-issue meets a
version that the earlier copy may hold, or may still be writing.
* `re-issuing a conditional storage write whose answer was lost`, with the
operation and the failure's class, for each re-issue of a control-object
write.
* `took over a silent owner's cluster lock` when a restarted server takes over
its predecessor's lock, and an error when an owner's lease lapses or another
server claims it.
## Rust error API
Rust embedders updating from 0.14.0 must pass the publication's graph commit ID
before the diagnostic message to
`OmniError::manifest_publish_in_doubt(graph_commit_id, message)`. The ID must
name the attempted publication; do not substitute a storage path or invent an
ID when the outcome is unknown.
`ManifestError.publication_in_doubt` changes from `bool` to
`Option>`: `Some` carries that commit ID, and `None` means this is
not an in-doubt publication. Prefer the constructor over struct literals, and
use `is_manifest_publish_in_doubt()` or `publication_id()` to inspect an
`OmniError`. HTTP error responses and CLI syntax are unchanged.
# Troubleshooting
Source: https://www.omnigraph.dev/docs/operations/troubleshooting
CLI failures include a human-readable message. Add `--json` where supported to
receive structured fields suitable for automation. Every HTTP error the server
answers, an unknown route, a method the route does not serve and a malformed
body, query string or path parameter included, uses a JSON body with `error`,
`code`, `disposition` and optional details for conflicts, limits, Blob ranges,
external sources, or recovery. A malformed request is `400`, a body without a
JSON content type `415`. MCP tool calls report the same body beside the HTTP
status ([MCP](https://www.omnigraph.dev/docs/operations/mcp.md#results-and-errors)).
Do not parse human-readable error text when a structured field is present.
## Server discovery
Remote graph commands first make an anonymous `HEAD /healthz` request to the
configured service root. Discovery sends neither the data bearer token nor URL
Basic-auth credentials; reverse proxies must allow that public probe. A timeout
or connection failure is reported with a credential-safe cause category and
means the data request was not sent. HTTP discovery refusals retain their status.
## HTTP errors
Each `code` is sent under the statuses `ErrorCode` in `openapi.json` names
beside its meaning: one, except `branch_start_refusal`, which is `409`, or `410`
for `not_retained`.
| Status | Meaning | Usual action |
| -----: | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 400 | Invalid request, query, schema, configuration, or external-Blob policy, including an update that must carry a stored external Blob reference the policy does not admit | Correct the request; retrying unchanged will fail again. For a stored reference, assign that property in the same update (a new value or null), or admit its base in the policy |
| 401 | Missing or invalid bearer token. `WWW-Authenticate` says which: `error="invalid_token"` when the server refused the token sent, and `resource_metadata` when the server accepts OAuth sign-in | Supply a token configured by the server. The CLI sends a token only to the server it is keyed to; see [server credentials](https://www.omnigraph.dev/docs/cli.md#use-a-server) |
| 403 | The resolved actor is not authorized (`forbidden`), or a valid OIDC token's subject is not in the server's admission file (`not_admitted`) | For `forbidden`, change policy or use an authorized identity. For `not_admitted`, signing in again cannot help: the server's operator must admit the identity |
| 404 | Graph, query, branch, entity, or route is unavailable | Check the name and applied cluster revision; stored-query denials may also appear as 404 |
| 405 | The path is served, but not with this method (`method_not_allowed`) | Use a method the `Allow` header lists |
| 408 | The request body did not arrive before its deadline (`request_timeout`) | Send the request again; nothing was admitted |
| 409 | Concurrent change, duplicate ID, merge conflict, existing resource, incompatible full-text index, a branch whose writer epoch refuses the writing process (`writer_fenced`), or a commit no branch can start at (`branch_start_refusal`) | Inspect structured details; not every conflict is retryable |
| 410 | Required change-feed history was reclaimed, or the commit a branch start names (`branch_start_refusal`, `not_retained`) | Capture and durably install a new baseline, then resume from its terminal cursor; start the branch at a retained commit |
| 412 | Blob entity-tag or graph-commit precondition failed (`precondition_failed`), or `Omnigraph-If-Actor` named another actor than the credential (`actor_mismatch`) | For `precondition_failed`, refresh the Blob ETag (a Blob write's `blob_precondition_failure` names the current one), or re-read the branch and retry the mutation with its current graph commit. For `actor_mismatch` nothing ran: use the named actor's credential or correct the header; retrying unchanged fails again |
| 413 | Request or operation exceeded a bounded resource limit (`payload_too_large`) | Split or reduce the operation using the reported limit. Blob limits are listed in [Blob limits](https://www.omnigraph.dev/docs/schema/blobs.md#limits) |
| 415 | The body's content type is not one the route accepts (`unsupported_media_type`) | Send JSON with `Content-Type: application/json`, or the type the route names |
| 416 | Blob byte range is outside the value (`range_not_satisfiable`) | Use the returned length to choose a valid range |
| 424 | An allowed external Blob source could not be read | Restore source availability or correct its URI/credentials |
| 429 | Server or per-actor admission limit reached | Use the whole-command disposition below before retrying; preserve `Retry-After` |
| 500 | Server or stored-data integrity failure | Check server logs; do not assume partial success. A Blob delivery, change-route or streamed-response failure logs its error class (`error_variant`, and `storage_kind` for a storage failure), never its storage path; see below |
| 503 | Admission is closed, the server's OIDC identity snapshot has expired, or the graph is not served (`service_unavailable`, `graph_unavailable`, both with `Retry-After`), or a published schema change requires completion (`recovery_required`) | Inspect the structured error; generic 503 is not permission to repeat a write |
A `GET` or `HEAD /blob` 500 logs `error_kind="blob_pre_header_internal"` with a
`stage`. `target` is a failure resolving a snapshot target for a policy-gated
request. `cell` is any failure of the engine's Blob read, including the branch
or snapshot resolution the engine does itself. `transport` is the server's own
refusal before headers, logged with `error_variant="unclassified"`. A managed
Blob body that fails after the headers logs the byte range under one of
`blob_payload_read` (with the error class), `blob_payload_short_read` (with
the returned and expected byte counts) or `blob_payload_permit_closed`.
A change route (`GET /changes`, `POST /changes/baseline`,
`GET /commits/{commit_id}/changes`) answers an internal failure with a fixed
500 and logs `error_kind="change_route_internal"` with `error_variant`,
`storage_kind` and `manifest_kind`. A change route that requires recovery logs
the `operation_id` the 503 reports. An export or change-baseline response that
fails after its 200 headers ends with an incomplete body and logs
`error_kind="served_stream_failed"` with `stream` (`export` or
`change_baseline`) and the same error class. None of these logs carries the
error's message, which can hold object URIs or credentials.
A graph-head `412` includes `precondition_failure` with `expected` and, when
available, `actual`. A change-feed `410` includes `change_feed_gap`; retrying
the same cursor cannot recover its missing history.
A `400` that refuses a query includes `diagnostic`: the stable code (`Q…`
parse, `T…` typecheck), `position` (`line`, `column`, `byte`) for a parse
refusal or `stage` (and `expression` when known) for a later one, `expected`,
and `fix` when one exists. Act on the fix; a retry of the same source fails
the same way. See [Diagnostics](https://www.omnigraph.dev/docs/queries-search/diagnostics.md).
## Failed data-write commands
Every server error body, and every CLI data-write and read failure, carries a
`disposition`: `effects` is `none`, `partial` or `unknown`, and `action` is
`fix`, `refresh`, `retry`, `recover` or `reconcile`. The server decides it for
the request it answered by the first row that matches:
| Error | `effects` | `action` |
| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- | ------------------------- |
| Any error before the request submitted a write | `none` | the refusal's own |
| A write whose completion the server could not settle; `code` becomes `outcome_unknown` unless the action is `recover` | `unknown` | `reconcile`, or `recover` |
| A refusal before the write started (`401`, `403`, `412`, `429`, `api_contract_mismatch`, `actor_mismatch`), a typed refusal of its one publication (a merge, key, read-set or dataset-version conflict, a writer-epoch refusal, a resource limit, an external Blob source, a query diagnostic), or a failure the engine proved published nothing, such as a storage failure before publication; `code` is kept | `none`, or `partial` when an earlier step of the write published, such as a load that forked its missing branch | the refusal's own |
| Any other error after a write, which may follow its publication; `code` is kept | `unknown` | `reconcile`, or `recover` |
A refusal's own action follows its code: `recover` for a recovery intent or a
full-text index to build, `retry` for `too_many_requests`,
`service_unavailable`, `graph_unavailable` and `request_timeout`, `fix` for
`unauthorized`, `bad_request`, `api_contract_mismatch`, `actor_mismatch`,
`not_found`, `method_not_allowed`, `unattached_cannot_publish`,
`branch_start_refusal`, `payload_too_large`, `unsupported_media_type` and
`range_not_satisfiable`, and
`refresh` for the rest: `forbidden`, `precondition_failed`, `conflict`,
`writer_fenced`, `external_blob_source`, `change_feed_gap` and `internal`. A write the server
could not settle answers `500`.
The CLI reports the server's disposition when that request was the command's
only effectful one, and otherwise its own account of the whole command, so a
lost response is `unknown` with `reconcile`. A read command changes nothing, so
its failure always has `effects: none`. JSON mode writes the failure to stdout,
or to stderr when part of the result, such as a streamed page, already reached
stdout. Stdout then keeps the bytes already written, which can end inside an
unfinished object, as for `diff`, `branch diff`, `commit changes` and
`changes poll` with `--json` when they follow every page, and the failure
document is on stderr. Human mode prints the `disposition` line on
stderr. A target refusal on these commands exits 1 with `action: fix`. HTTP
failures also retain `http_status` and, when present, the original
`retry_after` string.
This includes `commit publish` and `commit drop`, as well as their `mutate`
statement forms. If a response or result output is lost, inspect the commit's
current state before another attempt; an output failure does not undo the write.
The same applies to `load`, `blob put`, `blob clear` and `schema apply`: a closed
output pipe or failed result flush can follow a successful write. The CLI exits with a failure
and does not repeat the operation. If stdout cannot carry a JSON failure, stderr
reports the undelivered diagnostic and its disposition, and the exit still
follows it.
For the data commands the CLI classifies (queries, reads, `load`, `mutate`,
branch, commit, schema and Blob commands, and aliases), the
exit follows from the disposition:
| CLI exit | What to do |
| -------: | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| 0 | The operation succeeded or was a no-op. A merge can separately report a source-deletion failure without losing its successful result. |
| 1 | `effects: none`: the command changed nothing. Inspect the action and details, correct the cause and run it again. |
| 2 | `effects: unknown` or `partial`: the command may have had effects, or an earlier step of it did. Read the target, and reconcile or recover as `action` says, before another attempt; a conflict, resource limit, incomplete response or lost connection does not by itself permit repeating the write. |
| 4 | A condition you set did not hold, with no earlier command effects: a verified write precondition or an `--as` assertion. Re-read the branch and choose a new precondition deliberately. |
| 75 | `effects: none` with `retry`: the server refused before starting, as when the actor is busy. Honor `Retry-After` and bound any caller retry. |
| 64 | The argument parser refused the command line before anything ran (`EX_USAGE`); an addressing refusal inside a command, such as two scope flags, exits 1. |
Maintenance commands (`init`, `upgrade`, `optimize`, `index build`, `index rebuild`,
`cleanup`, `schema upgrade-system-columns`), `cluster`, `policy`,
`target`, `credentials`, `mcp`, `studio` and the managed commands keep their own exits for now:
their exit 1 does not by itself say that nothing changed, so read their report;
`mcp` answers each relayed request with its own disposition.
A served write refused with `401` `unauthorized`, `403` `forbidden` or `400`
`api_contract_mismatch` exits 1 with `effects: none`, and `action: refresh` for
the policy denial and `fix` for the others. The server answers these before it admits the write:
bearer authentication, then the API contract header, then policy. Fix the
credential, the policy grant or the client before running the command again.
If a server target with `auth: {provider: anonymous}` receives a 401, the CLI
adds `missing_credential` with `target`, `auth: "anonymous"` and a setup `hint`.
A missing credential for a token or exec provider refuses locally before any
HTTP request. The server's `error` text is unchanged.
The status counts only with its matching `code` in a response that carries the
server's API contract header: a bare `403` from a proxy stays `unknown` with
`reconcile`, and a refusal that also reports recovery work stays `recover`.
`fix` means correct the credential, input, path or client; sent unchanged, the
command fails the same way. `refresh` means read the current state and decide
again; it does not grant automatic replay. `partial` effects mean an earlier
step stands although the failing one wrote nothing; read the target before
deciding. `recover` means finish the reported recovery obligation.
`reconcile` means establish the original operation's outcome before another
attempt. Only `retry` with exit 75 grants the bounded retry described above; the
CLI performs no automatic retry.
The disposition covers the entire command. A later refused subrequest cannot
undo an earlier branch creation, load or publication. An embedded write's own
refusal says what the engine did, projected as a server would project it: a
policy refusal precedes every publication, a `load --from` fork included, so it
has `effects: none` on both planes. The engine marks an error that follows a
fork which did publish: a typed refusal then has `effects: partial` on both
planes, and an error the engine cannot prove wrote nothing has `unknown`. A
`load --from` onto a branch that already exists forks nothing. Any later
failure, such as rendering a committed receipt, stays `unknown`.
Disconnecting an admitted HTTP write does not cancel it: the server keeps
running the original operation, and a lost response can leave the caller's
outcome unknown. There is no durable request-result lookup in this release.
## Conflicts
A `409` is not one universal retry signal:
* A read-set or version conflict means another writer changed an input. Inspect
the whole-command disposition and re-read before deciding on a new operation.
* A key conflict means strict insertion found an existing ID. Change the ID or
use merge/upsert semantics; repeating the strict insert is not useful.
* A merge conflict requires an explicit resolution on one branch before
merging again.
* A full-text incompatibility includes
`full_text_index_rebuild_required: { "index": "…", "reason": "…" }` with
`code: "conflict"`. This condition persists until an operator rebuilds the
affected branch's indexes; do not automatically retry the same query. Follow
the [full-text upgrade procedure](https://www.omnigraph.dev/docs/operations/upgrade.md#full-text-index-upgrade).
* A full-text index with no part at the query's snapshot includes
`full_text_index_required: { "index": "Type.property" }` with
`code: "conflict"`. It persists until the branch's index is built
(`omnigraph index build --target --branch `); do not retry
the same query. See
[Search indexes](https://www.omnigraph.dev/docs/queries-search/search.md#indexes).
* “Already initialized” means the target already contains a graph. Choose a new
root or deliberately use the command's destructive option when appropriate.
* `code: "writer_fenced"` means the branch's [writer epoch](#writer-epoch)
refused the writing process. Repeating the request to that process is
refused again.
### Writer epoch
A branch at storage format v17 carries a writer epoch, a number that starts at
0\. A claim raises it by one and makes the claiming process the branch's
owner; a claim by the process that already holds the epoch writes nothing.
From then on, every publication to the branch, which is how a mutation, a
load, a merge into the branch and a schema apply land, is refused unless the
writing process holds the branch's current epoch. A process whose own claim
another process has since replaced is refused like one that never claimed.
The owner serves reads of a held branch its handle is bound to from its own
last publication, without checking the branch first. So a replaced owner can
keep answering reads from before the claim until a write of its own is refused.
The refusal is HTTP 409, `code: "writer_fenced"` and
`writer_fenced: { "held": …, "read": …, "current": … }`: `current` is the
branch's epoch, `held` the refused process's own and absent when it holds
none, `read` the epoch a refused claim read. Nothing was written. Deleting
the branch is not refused: an owner whose branch another process deleted
keeps answering reads from the deleted branch until its own next write
there.
No CLI command or HTTP route claims a branch in this release. Only a program
that embeds the engine does, through `Omnigraph::claim_branch_writer`, under
the [`branch_claim`](https://www.omnigraph.dev/docs/operations/policy.md#actions) action, and only on a branch at
storage format v17: below it the claim is refused and names
`omnigraph upgrade`. A branch nobody claimed stays at epoch 0 and accepts
every writer as before. A process holds its claim until it exits or another
process claims the branch, so an owner that restarts claims again before it
writes. The epoch of a claimed branch never goes back: after the owner exits,
every writer is refused until a process claims the branch again. The graph
records the epoch and not its holder, so no command names the process that
holds a branch. So `omnigraph-server` and direct CLI commands cannot write a
claimed branch in this release; write to it through the embedding program
that holds it. A branch deleted and created again under its name starts at
epoch 0.
Writes are atomic at the graph-commit boundary. A normal validation, conflict,
or limit error does not mean that a subset became visible. When
`recovery_required` identifies an already published schema change, its table
pins and accepted schema contract are durable together; only the process
view may still need refresh. Reopen the graph read-write or restart its
server rather than replaying the change. Legacy recovery sidecars require
the original release, as described under [maintenance failures](#maintenance-failures).
## Storage-format mismatch
If a graph was written by a different storage-format generation, the binary
refuses to open it and names the required release line. Follow the qualified
explicit routes in [Upgrading](https://www.omnigraph.dev/docs/operations/upgrade.md) to preserve graph identity and history;
normal open does not migrate or reset the graph.
Do not edit internal metadata or copy files from individual backing datasets
between graph roots.
## Cluster failures
* Run `cluster validate` before `plan` or `apply`.
* Apply refuses existing unmanaged roots and missing managed graphs that remain
declared. Restore authoritative data to keep a missing graph, or remove its
declaration to delete it; see [deployment boundaries](https://www.omnigraph.dev/docs/clusters.md#deployment-boundaries).
* A server restarted after a crash takes over its predecessor's leased S3 lock
once the predecessor's heartbeats have been silent for a lease and a half
(45 seconds by default). A lock with no lease (on a local or Azure root, or
an older binary's) requires prior-owner and accepted-I/O quiescence, exclusion of other
admissions/unlocks, and its exact ID; follow
[ownership transfer](https://www.omnigraph.dev/docs/getting-started/deployment.md#writer-topology).
* Directory boot reads `cluster.yaml` to resolve storage, but served graph,
query, and policy resources come from applied state. Submit configuration
changes with `cluster apply --target` for live activation.
* By default one graph that cannot open is quarantined while healthy graphs
serve. Use `--require-all-graphs` when partial startup is unacceptable.
* `external_blob_base_overlaps_storage_root`: an `external_blobs` base lies
inside, or contains, the cluster storage root that holds every graph and the
applied state. For a new declaration, move the base to a sibling prefix before
applying. Correct existing external-Blob bindings in the desired configuration
and apply them. An invalid applied binding keeps its graph blocked;
if every graph is blocked, startup fails with `cluster_no_healthy_graphs`. See
[External Blob references](https://www.omnigraph.dev/docs/clusters/config.md#external-blob-references).
* `external_blob_storage_root_uncomparable`: the cluster storage root is
spelled with a path component an external Blob base cannot express (an empty
component, or a percent sign in a local path), so a base of the same storage
kind cannot be proven to lie outside it. Moving the base does not help:
correct the declaration or remove the allow-list before applying. Graph roots
remain fixed. Validation refuses the base, and an invalid applied
binding keeps the graph blocked, as for an overlap.
See [Operating a cluster](https://www.omnigraph.dev/docs/clusters.md).
## Maintenance failures
If `optimize`, `index build`, `index rebuild`, `cleanup` or `upgrade` cannot
write or flush its output, the CLI returns a failure without repeating the
operation. An output failure does not undo completed work. Inspect the graph's
state before another attempt; for an interrupted upgrade, follow the
[upgrade recovery procedure](https://www.omnigraph.dev/docs/operations/upgrade.md).
* Recovery required: reopen the graph read-write or restart its server. A
graph carrying a sidecar from a release before 0.12 must first be opened
read-write with that release (`omnigraph snapshot ` is such an open).
* Foreign versions: `cleanup` lists Lance commits above a table's last linear
version under `foreign_versions`; no read or write uses them and no command
adopts them (below).
* Cleanup row with `error`: the collector's trace of that table did not
finish, so nothing of that table was deleted; fix the named cause and
rerun. Other tables are collected and the command exits 0.
* Azure admission failure: inspect the lease owner before using the admission
tool's break-glass flow.
### Foreign versions
A table has foreign versions when its Lance linear history carries commits
above the table's recorded last linear version
(`omnigraph.last_linear_version` on the registration). Every Omnigraph write
is a detached commit that a graph commit pins, so such a commit came from
something else writing the table directory directly. The condition is per
table.
* Queries, mutations, loads, merges, index builds, schema apply, optimize
and cleanup are unaffected: none of them resolves the linear HEAD, and no
command adopts a foreign commit.
* A confirmed `cleanup --keep N --confirm` run lists them per table, under
`foreign_versions` with `--json` and by name in the human report, and never
deletes a foreign version or its files. The preview without `--confirm`
lists nothing.
* The `repair` command that reported them as `foreign_drift` was removed; read
`foreign_versions` from cleanup instead.
There is no command to run before writing: the write path has no
precondition on a table's linear history. To discard the foreign commits,
export the graph and load it into a new one.
See [Maintenance](https://www.omnigraph.dev/docs/operations/maintenance.md) and [Deployment](https://www.omnigraph.dev/docs/getting-started/deployment.md).
## Useful diagnostics
```bash
omnigraph version
omnigraph snapshot --target ./graph.omni --json
omnigraph commit list --target ./graph.omni --json
omnigraph cluster status --target ./company-brain --json
```
For server requests, retain the HTTP status, response body, request path,
timestamp, and server logs. Redact bearer tokens and storage credentials before
sharing diagnostics.
# Upgrade from 0.13 to 0.14
Source: https://www.omnigraph.dev/docs/operations/upgrade-0.14
**Existing 0.13 graphs do not need rebuilding for 0.14.** The main work is
updating configuration, commands and clients.
## Before upgrading
Finish pending deployments and resolve uncertain writes with the old CLI.
Stop application traffic, servers and writers. Preserve the old executable,
operator home and journals, and verify a backup of the **whole graph or cluster
storage root**. Upgrade the CLI, server and SDK together; do not mix versions
against one root. Rollback after new writes means restoring that backup.
Preserve each deployment folder's `.omnigraph` directory as well as the operator
home. Do not discard or rewrite unfinished-operation records during configuration
migration; follow the [recovery guidance](https://www.omnigraph.dev/docs/cli/targets.md#upgrade-from-the-old-format).
## Check graph and data compatibility
| What you have | What to do |
| ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
| A graph written by released 0.13 | Keep its root and history. 0.14 reads its format 14; the next ordinary publication advances that branch to format 16 |
| A standalone graph from an older release | Check the [supported in-place upgrades](https://www.omnigraph.dev/docs/operations/upgrade.md#storage-upgrade); unsupported formats require export/import |
| A cluster graph below format 14 | Follow the [rebuild and cluster cutover procedure](https://www.omnigraph.dev/docs/operations/upgrade.md#cluster-cutover) using an export from its original binary |
Do not run `init` on an existing graph or rebuild it just to upgrade the CLI.
When a rebuild is necessary, export with the old binary and load into a **new**
root. Export/import preserves entities but does not preserve commit history or
branch ancestry; retain the old root until cutover is verified.
For a standalone local graph, this offline check reports its upgrade status
without changing data:
```bash
omnigraph upgrade --target ./graph.omni --check --json
```
The explicit `upgrade` command does not support cluster graphs. Normal 0.13
graphs already open in 0.14; see [storage upgrade](https://www.omnigraph.dev/docs/operations/upgrade.md#storage-upgrade)
only if you need explicit stamping before using unpublished commits or unattached
branches, or are converting an older standalone graph.
## Rebuild operator configuration
Back up `~/.omnigraph/config.yaml` (or the file in your `OMNIGRAPH_HOME`), then
replace its old routing sections. **There is no automatic importer.**
Before:
```yaml
servers:
prod:
url: https://graph.example.com
defaults:
server: prod
default_graph: knowledge
```
After:
```yaml
version: 1
targets:
prod:
kind: server
endpoint: https://graph.example.com
default_graph: knowledge
auth: {provider: token, source: {store: {}}}
defaults:
target: prod
```
Use your actual endpoint and graph name. Install its token again:
```bash
printf '%s\n' "$SERVER_TOKEN" | omnigraph credentials set --target prod --stdin
```
Old resource profiles become targets too. `login NAME` and implicit token
environment overrides are gone; credentials must have an explicit source.
For other target kinds and managed sign-in, use the
[configuration guide](https://www.omnigraph.dev/docs/cli/targets.md).
### Local graphs and defaults
One-off local commands need no configuration: pass `--target ./graph.omni`.
To keep a named local destination, add it under the same `targets` mapping:
```yaml
local:
kind: direct
scope: graph
uri: file:///srv/graphs/dev.omni
```
Use an absolute `file://` URI in the file. Set `defaults.target: local` to use it
when `--target` is omitted; an explicit target overrides that default. A direct
graph target already identifies the graph and must not include `default_graph`.
Old `operator.actor` defaults are gone: pass `--as ACTOR` explicitly on supported
direct writes.
### CI credentials
For CI, replace the server target's `auth` value with an explicit environment
source and inject that variable through your secret manager:
```yaml
auth: {provider: token, source: {env: OG_SERVICE_TOKEN}}
```
This replaces the stored-token source; it does not need `credentials set`.
`OMNIGRAPH_BEARER_TOKEN` and per-name token variables are no longer read implicitly.
Set `OMNIGRAPH_HOME` for the job, or select its YAML with
`omnigraph --operator-config /path/to/config.yaml ...`. Keep tokens out of YAML.
## Replace CLI invocations
Prefix these commands with `omnigraph`:
| Before | After |
| ------------------------------------------------------ | ------------------------------------------------------------------------------- |
| `query people --store ./graph.omni --query queries.gq` | `query people --target ./graph.omni --query queries.gq` |
| `load ./graph.omni --data batch.jsonl --mode merge` | `load --target ./graph.omni --data batch.jsonl --mode merge` |
| `query people --server prod --graph knowledge` | `query people --target prod --graph knowledge` |
| `query people --profile NAME` | `query people --target NAME` after recreating that binding as a target |
| `alias experts databases` | `alias experts --params '{"topic":"databases"}'` after updating the alias below |
`init ./graph.omni --schema schema.pg` is unchanged and creates a **new** graph;
do not run it to upgrade an existing graph. Local paths can be targets directly;
remote endpoints need a named target. Folder context no longer selects a target.
Old direct `az://` CLI routes have no inline `--target` equivalent, and old
direct `s3://` routes keep one only for a cluster root's deployment, recovery
and maintenance commands ([S3 cluster roots](https://www.omnigraph.dev/docs/operations/s3-cluster-roots.md)).
Use a configured server for other graph operations; direct-only maintenance
does not gain a remote equivalent. See [capability boundaries](https://www.omnigraph.dev/docs/cli/targets.md#transport-and-capability-boundaries).
Aliases now bind to a target. Replace their `server`/resource binding with
`target`, remove `args` and `format`, and pass parameters by name:
```yaml
aliases:
experts:
target: prod
graph: knowledge
query: find_experts
params: {limit: 20}
```
Use your actual stored read query and its parameters. An alias needs its own
`graph` even when the target has `default_graph`; aliases never run mutations.
Choose output with `--format` or `defaults.output`.
Keep your `cluster.yaml` source bundle. `--config` selects its directory and
`--target` selects the destination. For the server target above:
```bash
omnigraph cluster validate --config ./company-brain
omnigraph cluster plan --config ./company-brain --target prod
omnigraph cluster apply --config ./company-brain --target prod
```
For local bootstrap or managed deployments, follow the
[cluster guide](https://www.omnigraph.dev/docs/clusters.md#bootstrap-a-cluster) or
[managed deployment guide](https://www.omnigraph.dev/docs/cli/managed-lifecycle.md#prepare-and-apply-configuration).
## Update SDKs and scripts
Install the matching TypeScript SDK alongside the server:
```bash
npm install @modernrelay/omnigraph@0.14.0
```
If you use the npm MCP package, upgrade it to `0.14.0` too. Custom HTTP clients
must send `Omnigraph-Http-Api: 0.14` and update their generated bindings;
changing only the header is insufficient. Refresh cached MCP tool schemas and
regenerate saved query plans.
| Consumer | Required change |
| -------------------- | ------------------------------------------------------------------------------- |
| Actor fields | Read `actor_id` instead of `actor` |
| `load --json` | Stop reading `uri`; use the invocation's target. Receipts include `base_branch` |
| Write failures | Read `code` and `disposition`, replacing `command_outcome` |
| Commit-list requests | Use `listing=` instead of the old `unpublished` / `unattached` booleans |
Use explicit `--json` for machine-readable write receipts. For classified data
commands, exit 2 means partial or unknown effects, 4 a failed condition, 64 a
parser refusal and 75 busy; exit 1 means no effects. Other command families have
their own exits. Inspect the reported disposition before retrying a write; a
lost response does not prove failure. See [failure handling](https://www.omnigraph.dev/docs/operations/troubleshooting.md#failed-data-write-commands).
## Check schemas, search and embeddings
* String properties used by full-text calls now need `@index`, for example
`text: String @index`. Update affected declarations and validate stored queries
before deployment. Compatible full-text indexes stay usable; if search reports
`FullTextIndexRebuildRequired`, follow the [index rebuild procedure](https://www.omnigraph.dev/docs/operations/upgrade.md#full-text-index-upgrade).
* Embeddings created by the older offline concatenation pipeline need regeneration
and reloading. Each spec must name the single source field declared by `@embed`:
```bash
omnigraph embed --input raw.jsonl --output embedded.jsonl \
--spec embeddings.json --reembed-all
```
This changes a file, not the graph. Review the output and reload it using the
intended load mode and preserved entity IDs; see the [offline pipeline](https://www.omnigraph.dev/docs/queries-search/embeddings.md#offline-file-pipeline).
* Review `@range` constraints before reapplying schemas: upper-only bounds are
inclusive, list bounds apply to each element, and out-of-range writes now refuse.
See [constraints](https://www.omnigraph.dev/docs/schema.md#constraints).
* Send branch names exactly: surrounding whitespace and control characters now
refuse instead of being trimmed.
## Validate before resuming traffic
```bash
omnigraph version --json
omnigraph config validate
omnigraph target inspect prod --resolved
omnigraph query people --target prod --graph knowledge
```
Use an existing read query for the last command. Configuration validation and
target inspection are offline; the query checks actual access. Verify a controlled
write in staging, representative searches, and scripts that parse results and
errors before restoring traffic. Keep the old installation and verified backup
until the new fleet is proven.
# Upgrading OmniGraph
Source: https://www.omnigraph.dev/docs/operations/upgrade
Normal open accepts storage formats v14 to v17 and never migrates a graph: a v14 or v15 branch
is stamped v16 at its first publish, and only the [storage upgrade](#storage-upgrade) stamps an
existing graph v17. A standalone
graph at v8 or v9 (release 0.11.x) or at v13 is converted in place by that offline upgrade. A graph at any other
format, and every cluster-managed graph below v14, is refused and takes the
[export/import rebuild](#rebuild) with the binary that wrote the graph.
Start with the [0.13 → 0.14 checklist](https://www.omnigraph.dev/docs/operations/upgrade-0.14.md) for fleet,
CLI configuration, HTTP/MCP, and client compatibility changes.
## Storage upgrade
`omnigraph upgrade` converts a standalone v8, v9 or v13 graph to v17 in place, and stamps a v14, v15 or v16 graph to v17 in place by the route `stamp-v14-to-v17`: one metadata-only commit per live branch still below v17, main first, and nothing else (no pending marker, nothing under `__history/legacy/`, no table row rewritten); stop every reader and writer and take the backup as for any upgrade, and a run that stops part-way is finished by running it again.
Branches, commit ids, commit history and table data are kept; no table row is
rewritten. Only main builds from 2026-10-01 to 2026-10-04 wrote v13.
Graphs at v14 or v15 already open in 0.14; an offline stamp is not a prerequisite
for ordinary reads and writes. The first ordinary 0.14 publication stamps that
branch v16, including in a cluster. Opening the graph or running a no-op does not
advance the stamp. `as commit`, `commit
publish` and `commit drop` refuse a branch still at v14; `as unattached branch`
refuses a branch still at v14 or v15. A standalone graph can use the offline
command before using those features. Cluster graphs must first have an ordinary
0.14 publication on the affected branch; `upgrade` cannot stamp cluster graphs.
A v16 graph, written by 0.14.x or 0.15.0, opens as it is and its branches stay
at v16: no publication stamps a branch v17. Only `omnigraph upgrade` does, with
every process on the graph stopped. A binary that serves up to v16 then refuses
the graph, so upgrade every binary first. A graph this binary creates is born
at v17. A cluster graph created by an earlier binary stays at v16, since
`upgrade` cannot stamp cluster graphs.
The explicit `omnigraph upgrade` command is offline, and it cannot verify that itself:
1. Stop every server, embedded reader or writer and maintenance process that
has the graph open, and keep them stopped until the upgrade reports
`completed`. An old process opened earlier can still write to a branch; the
upgrade then stops and the root is restored from the backup.
2. Take and verify a backup of the whole graph root. There is no reverse
conversion: rollback is restoring that backup with the old binary.
3. Check, then convert, then verify with the new binary:
```bash
omnigraph upgrade --target ./graph.omni --check --json
omnigraph upgrade --target ./graph.omni --json
omnigraph commit list --target ./graph.omni --json
```
`--check` writes nothing. For a v8, v9 or v13 conversion, a run without `--check`
first marks the graph as pending, which every normal open refuses. It writes the
pre-upgrade commits under `__history/legacy/`, converts each live branch, validates
the result and only then removes the marker. Resume an interrupted conversion by
running the same command with the same executable. Never delete the marker or
anything under `__history/`.
| `outcome` | Meaning | Exit |
| ------------------- | ------------------------------------------------------------------------------- | ---- |
| `check_passed` | v8, v9, v13, v14, v15 or v16 graph, every check passed, nothing written | 0 |
| `already_current` | v17 graph, no live branch below v17, nothing to do | 0 |
| `completed` | converted, validated and served as v17, or every live branch stamped v17 | 0 |
| `check_failed` | a finding below; nothing was written | 1 |
| `recovery_required` | the graph is pending or holds leftover recovery files; follow `recovery.action` | 1 |
The report's `work` object counts what the run read and writes: `live_refs`,
`retired_refs` (deleted branches whose commits are kept), `orphan_writers`,
`legacy_commits`, `bookkeeping_versions`, `absent_parents`, `data_files`,
`id_shards`, `writer_shards`, `schema_contents`, `census_reads`,
`census_cells`, `legacy_bytes` and `table_opens` (the tables of a v8 or v9
graph opened to compare their columns with the root schema objects). A
resumed run that finds the legacy objects complete reads no history again and
reports `retired_refs`, `orphan_writers`, `bookkeeping_versions`,
`schema_contents`, `census_reads`, `census_cells`, `legacy_bytes` and
`table_opens` as zero.
| Finding | Names | What to do |
| ------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `unsupported_target` | the target | `--to-format` accepts 17 only |
| `newer_than_binary` | the format | upgrade the binary |
| `unsupported_source` | the format, layout, over-budget branch, or the unreadable, over-bound or table-mismatched root schema objects of a v8 or v9 graph | [rebuild](#rebuild) with the build that wrote the graph. For a retired branch, instead of rebuilding run `cleanup --keep --confirm` with that build, then `upgrade --check` again; rebuild only when the retired branch stays. For root schema objects, restore the whole root from one backup, objects and tables together, then check again; the objects alone, from an older backup, no longer match the tables and are refused. An unreadable contract refuses the 0.11.x open too, so export with 0.11.x is no way out of it |
| `source_recovery_required` | leftover recovery files, or a `.staging` root schema object of a v8 or v9 graph, each with or without a live `__schema_apply_lock__` branch | stop every writer, then open the graph read-write with 0.11.x: `omnigraph snapshot ` is such an open (`upgrade --check` is not). The open removes the recovery files and finishes or rolls back a schema apply that changed the table set; a property-only `.staging` it refuses and names the manual choice between the live and the staging schema. The lock stays live after the open; the upgrade handles it (next row). Then check again |
| `schema_apply_lock_retired` | a live `__schema_apply_lock__` branch on a root with no `.staging` object and no recovery files: the lock of a v8 or v9 schema apply killed before it released it | nothing: the upgrade handles it, and the converted graph does not hold it as a live branch; `--check` reports it, the run handles it |
| `history_objects_present` | object keys | restore the whole root, `__history/` included, from the backup of the earlier attempt |
| `legacy_lineage_incomplete`, `legacy_lineage_corrupt` | up to 20 commit ids or branches | rebuild |
| `legacy_record_over_bound`, `legacy_census_over_bound`, `legacy_directory_over_bound` | the measured value and its limit | rebuild |
| `legacy_uncommitted_change`, `legacy_head_record_mismatch` | the branch, version and field | report it; rebuild meanwhile |
| `preflight_failed` | the error | fix the cause and check again; nothing was written. Above the live-branch limit: rebuild |
| `pending_upgrade`, `upgrade_interrupted` | the attempt | rerun without `--check` with the same executable |
| `unknown_upgrade_ownership`, `fence_publication_attempted` | the marker state | keep the root, run `--check`, finish with the executable that started the attempt |
| `legacy_plan_changed`, `legacy_objects_differ` | digests | finish with the executable that marked the graph; restore the backup if an object under `__history/legacy/`, or the schema archive under `__history/schemas/` that a fenced v8 or v9 rerun reads instead of the root schema objects, was changed |
Limits, checked before anything is written: 1,024 live branches including
main and 1,024 retired branches; 1,000,000 catalog rows or 64 MiB per branch
head, live or retired, and 64 MiB per root schema object of a v8 or v9 graph;
256 KiB of commit fields and 16 MiB per commit record; a history-read budget
that refuses a single v13 branch line above about 75,000 commits; and a
history-memory budget of 1 GiB that admits about 3.8 million table entries,
counted for each branch as its commits plus its head, times the tables of its
head (10,000 commits over 383 tables, or 1,000 commits over 3,800). That count
multiplies every commit of a branch by the tables of its head, so a graph
whose tables were added late is over-counted. The budget counts only the
fixed-size part of each table entry: table names and paths are not counted
and planning holds a second copy, so the process needs several times 1 GiB of
memory. A graph over a limit is rebuilt. A retired branch over a limit cannot
shrink; 0.11.x `cleanup --keep --confirm` removes it when no live branch,
merge base or tag needs it (without `--confirm` it is a dry run), then run
`upgrade --check` again.
After the upgrade, commit ids from before it resolve as before, and commit
listing, the change feed, merges and `cleanup` read across the upgrade. A
snapshot addressed by a catalog version below the upgrade serves the commit
at that version, or the nearest commit below it. The upgrade lands the graph
at v17, so a binary that serves only up to v16 refuses the upgraded graph
before any open: upgrade every binary first.
A graph written by 0.11.x (v8 or v9) has limits of its own. Its checks read every catalog version since `init`, so a long history is slow to check.
| Subject | After the upgrade |
| ---------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Root schema objects | `_schema.pg`, `_schema.ir.json` and `__schema_state.json` at the graph root are read as the graph's schema and must be present and consistent. The upgrade leaves them in place. v14 to v17 never read them, and they go stale at the next schema apply. `_graph_commit_recoveries.lance/` also stays, as an unreferenced dataset |
| Unfinished schema apply | A `.staging` copy of a root schema object, or a leftover recovery file, is refused as `source_recovery_required`, with or without a live `__schema_apply_lock__` branch: open the graph read-write with 0.11.x (`omnigraph snapshot `), which removes the recovery file and finishes or rolls back an apply that changed the table set. An apply that changed properties only leaves a `.staging` that 0.11.x refuses as well; its message names the manual choice between the live and the staging schema. 0.11.x releases the lock only inside the process that took it, so the lock stays live after that open. A live lock on an otherwise clean root is handled by the upgrade: the converted graph does not hold it as a live branch, and the next schema apply is not refused as already in progress |
| Schema of old commits | 0.11.x kept only the latest schema, so every commit from before the upgrade is recorded under the one schema live at the upgrade. The tables and rows of an old commit are exact. A read at an old commit id is planned against the live schema, as in 0.11.x: a type added later is absent there, and a type dropped later cannot be queried |
| Table versions above a commit | An interrupted 0.11.x write can leave them. The upgrade opens each table only at the version main's head pins and does not report them. Reads and writes ignore them and `cleanup` lists them under `foreign_versions` ([foreign versions](https://www.omnigraph.dev/docs/operations/maintenance.md#foreign-versions)) |
| `cleanup` after a 0.11.x `cleanup` or `schema apply --allow-data-loss` | Both removed table versions that old catalog versions still name. `cleanup` deletes no catalog version of a live branch, so a catalog version can outlive the table versions it names, after these 0.11.x runs and after any earlier `cleanup` whose policy was narrower. A `cleanup` whose policy retains such a catalog version reports `is absent from the listing` for its tables, deletes nothing from them and still exits 0. `--older-than D` passes once those versions are older than `D`, and `--keep N` once they fall outside the newest `N`. A branch base, a merge base and a tag are retained under every policy: while one of them is such a version, every policy refuses its tables, and deleting that branch or tag releases it. `omnigraph cleanup --keep 1 --confirm` retains only those and each branch head, so it passes unless one of them is such a version. A read at a commit id whose table version 0.11.x removed fails as it did before |
Select a standalone local direct-graph target with `--target`. Server, managed and
direct-root targets are refused, as is a direct graph inside a cluster layout.
Cluster-managed graphs below v14 require [rebuild](#rebuild) with an export from
their original build. A 0.13 cluster's v14 graphs open directly in 0.14 and need no
rebuild. Local paths, file URIs and symlinks resolve before the ownership check.
## Lance substrate upgrades (v0.10, v0.14)
v0.10.0 moved from Lance 9 to Lance 11 on graph storage format v6: no export/import
migration, but the full-text upgrade below. v0.14 moves to Lance 13 with graph
format 16 and V2\_2 data files unchanged: no `omnigraph upgrade` for a format-16
graph, and full-text indexes stay searchable, the analyzer being unchanged.
Neither transition is rolling-compatible. Update the CLI, server, and client
integrations together with application traffic stopped, never run an old and new
fleet against the same graph, and keep the old executables and a verified whole-root
backup until the upgrade is proven: a graph the new fleet has written is not qualified
for the old reader ([Lance contract](https://github.com/ModernRelay/omnigraph/blob/main/docs/dev/versioning.md#lance-contract), [v0.10 compatibility notes](https://github.com/ModernRelay/omnigraph/blob/main/docs/releases/v0.10.0.md#compatibility)).
## Full-text index upgrade
The v0.9-to-v0.10 transition changed the English stemmer while keeping graph
storage format v6. That historical procedure requires the matching old binaries;
see the [v0.10 upgrade guide](https://github.com/ModernRelay/omnigraph/blob/v0.10.0/docs/user/operations/upgrade.md#full-text-index-upgrade)
for its branch inventory, rebuild and rollback steps. The current CLI cannot open
v6, so a graph still on that format needs an [export/import rebuild](#rebuild).
In v0.14, certified full-text indexes from Lance 11 remain searchable with
Lance 13 because the analyzer generation is unchanged. Missing or incompatible
analyzer proofs still fail closed. For a graph that normal open accepts
(formats 14–17), use the explicit maintenance command on each branch that needs
new indexes:
```bash
omnigraph index rebuild --kind full-text --target ./graph.omni --branch main --as operator --json
```
Follow the [full-text maintenance procedure](https://www.omnigraph.dev/docs/operations/maintenance.md#rebuild-full-text-indexes)
for authorization, tokenizer changes and result checks. Rebuilding a live branch
publishes its indexes in one graph commit; it leaves earlier snapshots unchanged,
so those snapshots may still refuse full-text search. `optimize` is not a
replacement for an analyzer rebuild.
A retained published commit under the current schema can start a branch with
`branch create --from-commit `; pre-upgrade commits are refused. See the
[branch-start restrictions](https://www.omnigraph.dev/docs/branching.md#start-a-branch-at-a-commit).
### Unsupported index inventory
A missing physical index kind or unsupported index format makes the rebuild
refuse before publication. Use compatible old tooling for a controlled
[export/import rebuild](#rebuild), or restore a trusted backup with supported
inventory. Keep the source intact; do not guess an index kind or edit metadata
to bypass the refusal.
### Cluster state and rollback
For the historical v0.10-to-v0.9 rollback, including `external_blobs` state,
follow the [tagged rollback procedure](https://github.com/ModernRelay/omnigraph/blob/v0.10.0/docs/user/operations/upgrade.md#cluster-state-and-rollback)
with those binaries. It is not a downgrade path from v0.14.
## What an export/import rebuild preserves
This table describes rebuilding a graph at a new URI for a graph-format change,
not the full-text index rebuild above. An index rebuild retains the existing
graph's entities, branches, and history.
| Preserved | Starts fresh |
| -------------------------------------- | --------------------------- |
| Nodes and edges in the exported branch | Commit history |
| IDs and property values | Branch topology |
| Stored vectors | Old snapshots |
| Managed Blob values | Physical indexes and layout |
Export each branch you need separately. Each import becomes an independent
main branch in a new graph; shared ancestry is not reconstructed.
## Choose the export binary
The refusal message names the release line that wrote the graph. The known
mapping is:
| Storage generation | Export with |
| ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
| v1 | 0.3.1 or earlier |
| v2 | latest 0.6.x |
| v3 | latest 0.7.x |
| v4 | latest 0.8.x |
| v5 | the exact unreleased development build that wrote it |
| v6 | latest 0.10.x (the refusal names 0.9.x or 0.10.x) |
| v7 | the exact unreleased development build that wrote it |
| v8, v9 | latest 0.11.x (v8: legacy system column spellings). A standalone graph takes the [storage upgrade](#storage-upgrade) instead |
| v10 to v13 | the main development build that wrote it; the refusal names its build dates. A standalone v13 graph takes the [storage upgrade](#storage-upgrade) instead |
| v14 to v17 | current binary; entity export/import is not required within this storage generation |
If the graph's generation is newer than the binary, upgrade the binary instead.
## System-column upgrade (legacy spellings)
`omnigraph schema upgrade-system-columns --target ` respells a standalone graph's
legacy system columns in place (`id`/`src`/`dst` to `__id`/`__src`/`__dst`).
It needs a graph that normal open accepts. The respelling
changes no storage layout and publishes the replacement contract with the
renamed table references in one new graph commit. Columns are renamed by field id: no table
rows are rewritten, indexes survive, and data and existing history/commit IDs are preserved;
`--check` runs the preflight and writes nothing; `--json` prints the report.
The preflight refuses a graph with any non-main branch (merge what you need,
then delete the branches: a merge alone leaves the source live) and a
property whose name starts with `_` (rename it with `@rename_from` first),
naming every offender under `system_columns_preflight`. Edge constraints
such as `@unique(src, dst)` become `@unique(@src, @dst)`. Stop every server
serving the graph and retain a verified backup first. Concurrent graph movement
can refuse the attempt; stop writers and rerun from a fresh capture.
There is no reverse operation. A run interrupted before it publishes changes nothing: rerun
it. Once publication succeeds, fresh read-only and read-write opens use the
accepted schema; a rerun reports `already_current`. A graph that
already spells `__id`/`__src`/`__dst` reports `already_current`;
cluster-managed graphs are refused.
## Rebuild
Keep separate executables and target URIs so the source remains recoverable.
Stop all source writers, including embedded and standalone CLI writers, before
exporting schema and data, and keep them stopped through cutover. Take and verify
a whole-root backup; use the old binary for the exports below.
When an export already carries top-level `id`, the rewrite preserves it and any user property named `id`. For predecessor schemas, change endpoint constraint references from `src`/`dst` to `@src`/`@dst` before the new `init`. See [ingestion](https://github.com/ModernRelay/omnigraph/blob/main/docs/dev/ingestion.md#strict-graph-batch) for the envelope contract.
```bash
# Old binary
old-omnigraph schema show s3://bucket/graph.omni > schema.pg
old-omnigraph export s3://bucket/graph.omni > graph.jsonl
# Relocate predecessor export identities
jq -c 'if has("id") then . else .id = .data.id | del(.data.id) end' \
graph.jsonl > graph-current.jsonl
# New binary (schema.pg endpoint constraints use @src and @dst)
omnigraph init --schema schema.pg ./graph-new.omni
omnigraph load --mode overwrite --data graph-current.jsonl \
--target ./graph-new.omni
# Verify with the new binary
omnigraph snapshot --target ./graph-new.omni --json
omnigraph schema show --target ./graph-new.omni
```
For another branch:
```bash
old-omnigraph export --branch review s3://bucket/graph.omni \
> review.jsonl
jq -c 'if has("id") then . else .id = .data.id | del(.data.id) end' \
review.jsonl > review-current.jsonl
omnigraph init --schema schema.pg ./graph-review-new.omni
omnigraph load --mode overwrite --data review-current.jsonl \
--target ./graph-review-new.omni
```
## Verify before cutover
At minimum:
* compare entity counts by type;
* run representative queries and mutations in a staging copy;
* sample IDs, vectors, null values, and Blob values;
* rebuild or reconcile declared indexes with `optimize`;
* verify server policy, stored queries, and external Blob access in the target
cluster;
* keep the old graph read-only until the new fleet is serving successfully.
Embeddings are copied as stored vectors; they are not regenerated. If the model
changed, re-embed after the import.
External Blob references require the target graph's allow-list to admit the
same sources. A direct-store CLI has no cluster allow-list and therefore cannot
admit new external references. Rebuild such data through a configured cluster
server or an embedded host that installs the policy. See
[Blob values](https://www.omnigraph.dev/docs/schema/blobs.md).
## Cluster cutover
Cluster graph roots are derived as `/graphs/.omni`; a
graph declaration cannot point at an arbitrary replacement root. Choose one of
these cutovers:
* **New graph ID in the same cluster.** Add (for example)
`knowledge-next` with the desired schema, query, provider, and policy
bindings. Validate, plan, and submit `cluster apply --target --config ` so the running
owner creates and activates its derived root. Load through that server, verify
the new ID, back up what you need, and move clients. Removing the old declaration
and applying deletes its managed storage and all retained history.
* **Same graph ID in a parallel cluster root.** Copy the source bundle and keep
the original cluster untouched. Bootstrap a separate root following
[cluster bootstrap and ownership transfer](https://www.omnigraph.dev/docs/clusters.md#bootstrap-a-cluster).
Validate the bundle, then for a local root plan and apply with
`--target ./cluster-new --config ./new-bundle`; any `storage:` assertion must
match that selected root, and changing `storage:` alone never selects it. After direct apply, establish
that its owner and accepted I/O have settled, then release its exact retained
lock as that procedure requires. Start the new server against the new root,
load through its server target, verify, and move clients to it. This preserves
the public graph ID while changing the whole deployment artifact.
In either case, quiesce writers before export and cutover. External Blob
references must be loaded through a policy-aware server or embedded host after
the target allow-list is applied.
Do not run a mixed fleet of binaries that disagree on the storage format. Do
not edit internal metadata, overwrite the source root with `init --force`, or
copy files or backing datasets between graph roots.
# Engine catalogue
Source: https://www.omnigraph.dev/docs/queries-search/catalogue
The engine describes itself in GQ. Five built-in node types, spelled with
the `@` prefix that already marks engine-owned metadata (`$p.@id`,
`$e.@type`), hold the registries the running binary executes with: its
physical operators, its planner passes, its session settings, its storage
format and its logical plan kinds. The rows are derived from those
registries at query time, so the list a reader sees is the list the engine
runs on. Nothing is stored; a user schema cannot declare an `@` type.
```graphql
query operators() {
match { $o: @Operator }
return { $o.name, $o.inputs }
}
```
The declarations, one per type, are in [`catalogue.gq`](https://github.com/ModernRelay/omnigraph/blob/main/docs/user/queries/catalogue.gq);
ordinary `schema show`
prints only the graph's reusable schema source; `--json` includes the
built-in declarations in the separate `catalogue_source` field.
## Where to run it
| Scope | Command |
| -------------------------------------------- | -------------------------------------------------------------- |
| local binary, without a store or credentials | `omnigraph catalogue -e '' --json` |
| beside a graph's own types | `omnigraph query --target -e ''` |
| served | `omnigraph query --target --graph og_catalog -e ''` |
`og_catalog` is a reserved scope the binary serves without opening a store:
no branch, no snapshot, no graph commit in the answer, and `--branch` or
`--snapshot` is refused. The local `catalogue` command accepts `-e ''` or
`--query ` and optional `--json`. It does not read operator configuration,
including an explicit `--operator-config`, or access credentials or storage.
It uses the binary's default table presentation unless `--json` is supplied.
It accepts no target, profile, authentication profile, actor, graph, branch
or snapshot selector. Operator version 1 has no catalogue target kind or magic
`--target og_catalog` selector. Local `./og_catalog` and `file://` paths remain
ordinary store addresses. The prefix
`og_` is reserved for system graph ids: a deployment cannot create a graph
named `og_…`.
Inside a graph scope the catalogue types sit beside the user types, so one
query joins them:
```graphql
query jobs_naming_no_operator() {
match {
$j: Job
not { $o: @Operator $o.name = $j.uses }
}
return { $j.name }
}
```
A served read of `og_catalog` is authorized like a graph read: the policy
bound to `graph.og_catalog` in `cluster.yaml` decides, with the resource
`og_catalog` and no branch; without a bound policy the ordinary no-policy
rules apply: every credential is refused, unless the server sets
`OMNIGRAPH_ALLOW_READS_WITHOUT_POLICY=1`, which lets static tokens read.
See [Authorization and actors](https://www.omnigraph.dev/docs/operations/policy.md).
## What is refused
* `insert`, `update`, `delete` and `load` targeting an `@` type: the rows
are the engine's.
* `search`, `fuzzy`, `match_text`, `bm25`, `nearest` and `rrf` over a
catalogue binding: catalogue rows have no index. Scalar predicates,
projection, aggregation, ordering, limits and joins work as on any type.
* Under `og_catalog`: any stored type, any write, blob, branch, commit,
snapshot, export, change feed or stored query.
`explain` names the source: a catalogue scan reports `source: catalogue`,
reads `access sequential`, is never counted from a manifest and never
hydrated by row address.
## The five types
**`@Operator`**: one row per distinct physical operator name the plan runner
executes; `inputs` lists every number of children the operator admits. Two
plan node kinds that report one name (`Limit` and `Page` both report `Page`)
share one row. `Scan` admits `[0, 1]`: a table scan has no input, a
dependent id lookup has one.
**`@Pass`**: one row per planner pass, `position` its place in the order
`explain` reports passes. A run fires a subsequence of this order; the two
scan-finalization passes `key_to_id` and `scan_access` come last.
**`@Setting`**: one row per session setting: `kind` spells the legal values
(`enum(…)`, `integer(min..)`, `integer(min..=max)`, `text`), `default` the
value `show` reports before any `set`, `scope` whether any caller
(`request`) or only the hosting process (`process`) may set it. The
descriptive text of each setting stays in [Session settings](https://www.omnigraph.dev/docs/queries-search/settings.md).
**`@Engine`**: the one row describing the binary: `version` is its release
version, `current_internal_schema_version` the storage format of a graph it
creates, `min_supported_internal_schema_version` the oldest stored format it still
serves. A graph's own stored stamp is in `omnigraph snapshot`, not here.
**`@LogicalKind`**: one row per logical plan node kind with its routing
`coverage`: `routable` or `refused_by_name`. Coverage governs change-feed
and merge routing only. Read and load plans have their own admission checks.
## Inventories
Generated from a built CLI with
`python3 scripts/check-catalogue-docs.py --binary target/debug/omnigraph --write`;
the same script without `--write` fails when these tables drift from the
binary's rows.
| name | inputs |
| ---------------- | ------- |
| Aggregate | \[1] |
| AntiJoin | \[2] |
| ClassifyThreeWay | \[1] |
| ContainsJoin | \[2] |
| CrossJoin | \[2] |
| CsvSource | \[0] |
| DeltaSource | \[0] |
| Expand | \[1] |
| Filter | \[1] |
| FragmentWrite | \[1] |
| HashJoin | \[2] |
| HydrateByAddress | \[1] |
| HydrateColumns | \[1] |
| MetadataCount | \[0] |
| OuterReference | \[0] |
| Page | \[1] |
| Projection | \[1] |
| RankFuse | \[2] |
| RowCompare | \[1] |
| Scan | \[0, 1] |
| Sort | \[1] |
| SortMergeJoin | \[2] |
| Validate | \[1] |
| position | name |
| -------- | ----------------------- |
| 0 | resolve |
| 1 | predicate\_pushdown |
| 2 | aggregate\_pushdown |
| 3 | projection\_pushdown |
| 4 | fragment\_scope |
| 5 | address\_short\_circuit |
| 6 | late\_materialization |
| 7 | join\_algorithm |
| 8 | expand\_mode |
| 9 | access\_path |
| 10 | anti\_join\_strategy |
| 11 | key\_to\_id |
| 12 | scan\_access |
| name | kind | default | scope |
| ------------------------- | ----------------------------------------------- | --------- | ------- |
| ann\_nprobes | integer(0..) | 20 | request |
| change\_lineage | enum(off, on, verify) | on | request |
| engine | enum(v2) | v2 | request |
| explain\_dump\_dir | text | | process |
| history\_release\_bytes | integer(1024..=262144) | 262144 | request |
| load\_source\_max\_bytes | integer(1..) | 33554432 | request |
| merge\_lineage | enum(off, on, verify) | on | request |
| query\_memory\_bytes | integer(1..) | 157286400 | request |
| rrf\_plan | enum(auto, force\_prefilter, force\_postfilter) | auto | process |
| stage\_write\_concurrency | integer(1..=64) | 8 | process |
| traversal\_work\_limit | integer(1..=9223372036854775807) | 1000000 | request |
| write\_max\_bytes | integer(1..) | 33554432 | request |
| name | coverage |
| -------------- | ----------------- |
| Aggregate | refused\_by\_name |
| AntiJoin | refused\_by\_name |
| CsvSource | refused\_by\_name |
| DeltaSource | refused\_by\_name |
| Expand | refused\_by\_name |
| Filter | routable |
| FragmentWrite | refused\_by\_name |
| Join | routable |
| Limit | routable |
| MergeClassify | routable |
| MetadataCount | refused\_by\_name |
| Nearest | refused\_by\_name |
| OuterReference | refused\_by\_name |
| Projection | routable |
| RankFuse | refused\_by\_name |
| RowDiff | routable |
| Sort | routable |
| TableScan | routable |
| TextSearch | refused\_by\_name |
| Validate | refused\_by\_name |
# Conditional mutations
Source: https://www.omnigraph.dev/docs/queries-search/conditional-mutations
Use `--if-commit` when a mutation should apply only to the graph state that
the caller read:
```bash
# The JSON response includes the commit pinned for these rows.
omnigraph query find_person --query queries.gq --target ./graph.omni --json
omnigraph mutate update_person --query queries.gq --target ./graph.omni \
--params '{"name":"Ada"}' \
--if-commit --json
```
The condition compares the effective head of the target branch. Any
intervening commit invalidates it, including a commit that changed an unrelated
entity or type. On mismatch, nothing is written: the CLI exits with code 4
against a server and with code 1 on an embedded graph such as this `--target`
run, and JSON output contains `precondition_failure` with `expected` and
optional `actual` commit ids. Re-read the branch and decide again; do not blindly retry
the old mutation. The precondition is still checked when the mutation would
match no entities. A mutation that loses a race to another commit reports this
precondition failure, even when that commit would also make it break a
constraint such as an edge `@card` maximum.
Use the `graph_commit_id` returned with the original query rows. Fetching a
head id afterward creates a race between the read and the precondition. For the
HTTP header and dedicated conditional routes, see the
[HTTP server guide](https://www.omnigraph.dev/docs/operations/server.md#conditional-mutations).
# Diagnostics
Source: https://www.omnigraph.dev/docs/queries-search/diagnostics
Every refusal of a query carries four things: a stable code, where the failure
is, what was expected or violated, and one concrete fix. The reader is often
an agent that treats an error as the documentation it acts on, so the fix
names the construct to use, not the rule that was broken. A refusal with no
fix names the decision instead.
| Field | Meaning |
| ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
| `code` | Stable identifier: `Q…` from the parser, `T…` from the type checker, `P…` from the planner. A code's meaning is frozen; its message text may improve. |
| `position` | For a parse refusal: `line` and `column` (1-based, in characters) and the `byte` offset. |
| `stage` and `expression` | For a refusal after parsing: the stage (`typecheck` or `plan`) and, when the site can render it, the expression it refused. |
| `expected` | What was expected or violated, one line, without a position. |
| `fix` | One concrete fix, absent when `expected` names the decision. |
| `suggestion` | Optional source edits with `applicability`, plus `start`, `end` and `replacement` for each edit. |
The one-line `error` text is the code and the expectation in the legacy form
(`parse error: …`, `type error: T33: …`, `plan error: P002: …`); the other
fields travel beside it.
## The measured case
A declaration without its parameter list:
```text
query name {
```
```text
error[Q002]: parse error: expected `(`: a query declares its parameters even when it has none
--> line 1, column 11
fix: query name()
```
Earlier releases reported this at the file's first position as `expected
query_file`.
## Typed suggestions
The `fix` remains human-readable guidance. When the compiler can propose a
specific edit, `suggestion` carries it as data. For example, this source:
```text
query name { match { $p: Person } return { $p.name } }
```
receives this suggestion alongside Q002:
```json
{
"applicability": "machine_applicable",
"edits": [{ "start": 10, "end": 10, "replacement": "()" }]
}
```
Ranges use zero-based UTF-8 byte offsets in the exact original request source,
with an exclusive `end`. Equal offsets insert text. Ranges do not overlap;
apply multiple edits from the highest offset to the lowest. Do not apply an
edit to a source that has changed since the request.
`machine_applicable` means the compiler has enough evidence for this local
correction. `needs_review` means the caller must review the proposed change.
Neither value guarantees that the corrected query will pass every check;
validate it again. The compiler does not apply suggestions automatically.
Q002 offers the `()` insertion only when the edited file matches the grammar
without another missing-parameter recovery. Incomplete or ambiguous input can
still receive textual guidance while omitting `suggestion`. Missing suggestions
do not mean there is no possible fix.
## Where the fields appear
* **CLI, human formats** (`table`, `kv`, `csv`): the form above on stderr,
then `HTTP 400` when served and the `disposition` line, exit status 1, with no
colour codes and no backtrace footer.
* **CLI, machine formats**: `--json` and `--format json` print the API's error
body pretty with its `disposition`, `--format jsonl` prints it as one line, and
`--format toon` prints that same body as TOON. All of them go on stdout with
exit status 1. A served refusal keeps the server's `code` field
(`bad_request`); an embedded one has none.
* **HTTP**: a `400` that refuses a query carries the additive `diagnostic`
object in its error body. See
[HTTP errors](https://www.omnigraph.dev/docs/operations/troubleshooting.md#http-errors).
* **`omnigraph lint`**: a `Q000` finding whose message is the expectation at
`line , column `; type errors report their `T…` code. See
[Linting](https://www.omnigraph.dev/docs/queries-search/queries.md#linting).
* **`queries validate --json` and `cluster plan --json`**: each breakage or
query parse/typecheck failure carries the same diagnostic information as
`diagnostic` or `detail`, so a stored query the next release would refuse
is a pre-upgrade finding. Cluster `detail` uses the compiler shape: `message`
and a nested `stage` object. API and validation output use `expected` and
separate `stage`/`expression` fields.
```json
{
"error": "parse error: expected `(`: a query declares its parameters even when it has none",
"code": "bad_request",
"diagnostic": {
"code": "Q002",
"position": { "line": 1, "column": 11, "byte": 10 },
"expected": "expected `(`: a query declares its parameters even when it has none",
"fix": "query name()"
}
}
```
## Parse codes
| Code | Meaning |
| ------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Q001` | The source does not match the grammar at the reported position; `expected` names the grammar rules the parser could accept there. |
| `Q002` | A query declaration is missing its parameter list. |
| `Q003` | A settings statement is refused: unknown setting, value outside its row, or a process setting in a request. |
| `Q004` | A branch, show or explain statement is misplaced or malformed. |
| `Q005` | A declaration body is refused; `expected` names the construct. |
| `Q006` | The source nests too deeply, checked before the grammar runs: more than 64 levels of brackets and adjacent `not`s, or an expression deeper than 256 levels when each `and`/`or` of a chain also counts one. Write a long `or` chain over one property as `$p.name in [...]`. |
Type codes are listed where the construct they guard is described, in
[Query language](https://www.omnigraph.dev/docs/queries-search/queries.md).
## Planner codes
A planner code refuses a well-formed, type-checked query shape by design. It
is the caller's error, answered as a bad request with the diagnostic, never as
an internal error.
| Code | Meaning |
| ------ | ----------------------------------------------------------------------------------------------------------------------------------- |
| `P002` | An edge alternation or wildcard traversal has no finite traversal work limit. The fix sets `traversal_work_limit` before the query. |
| `P003` | The traversal work limit is outside `1..=9223372036854775807`, or a plan records it twice. |
| `P004` | An edge alternation or wildcard traversal was asked to run in CSR traversal mode; it runs in `auto` or `indexed` mode. |
`P001` and `P005` are withdrawn. A search order ranks the binding it names
whatever the `match` block declares first, so a ranking on a binding a
traversal reaches is no longer refused, and an `rrf()` whose arms rank two
bindings one traversal connects is the type error `T59`
([Search](https://www.omnigraph.dev/docs/queries-search/search.md#ranking-a-traversal-reached-binding)).
# Embeddings
Source: https://www.omnigraph.dev/docs/queries-search/embeddings
OmniGraph can turn query text into a vector for
`nearest($document.embedding, $text)`. Use the same provider and model that
produced the stored document vectors; vectors from different models are not
comparable.
Generated vectors are finite, nonzero, and L2-normalized before use. The target
`Vector(N)` property determines their required dimension.
## Providers
| Provider | Configuration |
| ------------------- | ---------------------------------------------------------------------------------- |
| `openai-compatible` | Default. Uses an OpenAI-compatible `/embeddings` endpoint; defaults to OpenRouter. |
| `openai` | Uses OpenAI directly. |
| `gemini` | Uses Google's embedding API. |
| `mock` | Deterministic local vectors for tests and development. |
For direct or embedded use, configure the provider with environment variables:
| Variable | Meaning |
| -------------------------------------------------------- | -------------------------------------------------- |
| `OMNIGRAPH_EMBED_PROVIDER` | `openai-compatible`, `openai`, `gemini`, or `mock` |
| `OMNIGRAPH_EMBED_BASE_URL` | Override the provider endpoint |
| `OMNIGRAPH_EMBED_MODEL` | Override the model id |
| `OPENROUTER_API_KEY`, `OPENAI_API_KEY`, `GEMINI_API_KEY` | Provider credential |
| `OMNIGRAPH_EMBED_DEADLINE_MS` | Total call deadline; default 60,000 ms |
| `OMNIGRAPH_EMBED_TIMEOUT_MS` | Per-request timeout; default 30,000 ms |
| `OMNIGRAPH_EMBED_RETRY_ATTEMPTS` | Maximum attempts; default 4 |
| `OMNIGRAPH_EMBED_RETRY_BACKOFF_MS` | Initial retry backoff; default 200 ms |
| `OMNIGRAPH_EMBEDDINGS_MOCK` | Force the mock provider |
The default OpenRouter model is `openai/text-embedding-3-large`. The direct
OpenAI default is `text-embedding-3-large`; Gemini defaults to
`gemini-embedding-2`.
## Outbound requests
Apart from its storage, embedding is the only outbound request the server
makes: it happens when a query's `nearest` takes text rather than a vector, and
goes to the provider endpoint with that provider's key. Stored vectors are never
re-embedded by the server, and loads do not call a provider. To run with no
outbound requests at all, bind no `embedding_provider` and leave
`OPENROUTER_API_KEY`, `OPENAI_API_KEY` and `GEMINI_API_KEY` unset in the
server's environment, along with `OMNIGRAPH_EMBED_PROVIDER`: a text `nearest`
then fails, and every other query is unaffected. Pass vectors as `nearest`
parameters instead. The mock provider also makes no request, but its vectors
are not real embeddings.
## Cluster configuration
Cluster-served graphs select a named provider in `cluster.yaml`:
```yaml
providers:
embedding:
default:
kind: openai-compatible
base_url: https://openrouter.ai/api/v1
model: openai/text-embedding-3-large
api_key: ${OPENROUTER_API_KEY}
graphs:
knowledge:
schema: knowledge.pg
embedding_provider: default
```
Inline API keys are rejected. `${ENV_VAR}` references are resolved when the
server starts, not when the cluster configuration is planned or applied.
## Schema annotation
Associate a vector with its source text:
```graphql
node Document {
slug: String @key
body: String
embedding: Vector(1536) @embed("body", model="openai/text-embedding-3-large") @index
}
```
The vector is the embedding of the record's `body` value exactly as stored,
and a text `nearest($d.embedding, $q)` embeds `$q` exactly as given. Neither
side adds a type name, a field label or any other template, and neither trims
whitespace, so a record whose `body` equals the query text embeds the same
input. The one difference a provider can see is the retrieval role: Gemini
receives `RETRIEVAL_DOCUMENT` for stored text and `RETRIEVAL_QUERY` for a
query, and the other providers receive identical requests. Vectors you compute
yourself must embed the source value the same way to be comparable with text
queries.
When `model` is recorded, a text `nearest` query is rejected unless the active
provider resolves to exactly that model id. Changing the recorded source or
model is not an in-place schema migration; rebuild or re-embed the data instead.
`@embed` does not populate the property during a load, even with an embedding
provider configured. Supplied vectors are preserved, omitted nullable vectors
remain null, and missing required vectors refuse the load. Supply vectors in
input data or prepare seed files with the offline command.
Successful HTTP and CLI JSON load results include `embedding_generation`:
`"unsupported"` means the load touched a node type with an `@embed` property;
`null` means it did not. This describes the load capability, including when
every vector was supplied. Human CLI output prints the corresponding guidance.
## Offline file pipeline
`omnigraph embed` reads and writes JSONL files; it does not mutate a graph.
```bash
omnigraph embed --input raw.jsonl --output embedded.jsonl --spec embeddings.json
```
The spec names, for each node type, the vector property to fill and the one
property whose value it embeds:
```json
{
"dimension": 1536,
"types": {
"Document": { "target": "embedding", "fields": ["body"] }
}
}
```
`fields` must name exactly one property, the source the target's `@embed`
declares; a spec that lists none or several is refused. The command does not
read the schema, so keep `target`, `fields`, and `dimension` equal to the
`@embed` declaration and its `Vector(N)`. The source's string value is embedded
unchanged, as described under [Schema annotation](#schema-annotation). A
processed record whose source is absent, null, or blank gets no vector: whatever
its target held, including a null or an empty list, is removed and counted as
cleaned. A source value that is not a string is refused.
By default it fills missing vectors. Use `--reembed-all` to replace selected
vectors or `--clean` to remove them. `--type` and `--select` restrict the
records processed. A seed manifest can be supplied with `--seed` instead of
separate input, output, and spec paths; its `embeddings` entry takes the same
shape as the spec.
# Explaining a load statement
Source: https://www.omnigraph.dev/docs/queries-search/explain-load
`explain` before a `load csv` statement returns the load's accepted plan
instead of running the load:
```graphql
explain load csv "s3://imports/knows.csv" into edge:Knows mode overwrite delimiter "|" header
```
The statement accepts every load option. It reads metadata, but no payload,
and publishes nothing. HTTP and MCP require `change` on the target branch
before the external Blob policy check or metadata probe. A source over
`load_source_max_bytes` refuses as `csv_source_bytes`, as a load would.
The answer is the planner's document, in the same rows a query's
[explain](https://www.omnigraph.dev/docs/queries-search/explain.md) uses: trees `logical_initial`, `logical` and `physical`
in pre-order, then the `plan` rows. A load plan has three nodes, from the
root down:
| `node` | `detail` fields |
| --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `FragmentWrite` | `table`, `version`, `mode`, `adapter`; the physical row adds `max_input_bytes`, `max_workers` |
| `Validate` | `table` and `checks`, every check of the load in the order it runs, which is the refusal precedence; the physical row adds `removal` and `max_workers`. The node is a pipeline breaker only while a check reads the whole delta after the rows are in (`retained_edges`, a `judged` `edge_ri`, a `scan` removal); otherwise every check is swept as the rows pass and settled in the judgment after the drain, the node streams, and `pipelines` holds one pipeline from the source through the write instead of two |
| `CsvSource` | `format`, `source`, `source_bytes`, `estimated_rows`, `delimiter`, `header`, `columns`, `table`, `version`; the physical row adds `max_partitions` (the streams that parse the source: one per 8 MiB of it, at most `stage_write_concurrency`), `max_workers`, `fallback` |
Each entry of `checks` carries `check` (`value`, `unique`, `edge_ri`,
`cardinality`, `retained_edges` or `source_cardinality`), its identity fields,
`consults` (the committed table it reads, or `null`) and, on the physical
node, `access`:
* `unique` carries `columns` and `role` (`key` or `unique`).
* `edge_ri` and `cardinality` carry `endpoint` (`from` or `to`) and
`node_type`.
* `retained_edges`, on a node overwrite, carries `edge_table` and `endpoint`.
* `source_cardinality`, on a node load whose type is the source of an edge
type with a non-zero `@card` minimum, carries `edge_table`, `min` and an
optional `max`: each loaded node's committed edges of that type are counted.
`access` is the path the check reads committed state by:
| `access` | Meaning |
| -------- | ------------------------------------------------------------------------------------------------------------------------------- |
| `delta` | the check reads no committed state: a `@key` group, or a non-key `@unique` group under overwrite, whose committed view is empty |
| `probe` | a filtered key lookup; uniqueness uses chunks of 8,192 keys |
| `scan` | one projected scan of the key columns, matched in memory |
Overwrite endpoint checks cost probes and scans using estimated input rows,
the consulted table's row count, and usable BTREE coverage at the accepted
version. A probe first pays its key list, one entry per input row, so an
unindexed endpoint check scans, and a partially or fully indexed table keeps
`probe` only while the key list is small against it. Unknown counts and equal
costs keep `probe`; statistics record `index_coverage` and both costs.
`retained_edges` and `source_cardinality` always use `probe`, and so does
`cardinality` except on an overwrite under a non-zero minimum: there it uses
`scan`, one projected scan of the replaced table's source column for a source
the new edges leave without one. An `edge_ri` check
also carries `probe`, where it reads the committed ids: `swept` when both
endpoint checks of the edge use `scan` and both node types carry one
`I32`, `I64` or `U32` `@key`, so the committed sets load while the source
parses and each parse partition probes its own rows; `judged` otherwise,
after every row is in. Either way the first missing endpoint in row order, `from` before `to`
within a row, is the refusal. The node's `removal`
says whether the committed ids an overwrite removes are found at all: `scan`,
one projected scan of the committed id column, when a `retained_edges` check
reads them; `none` when no planned check does. A node target whose committed
table holds no rows removes nothing, so its plan carries neither the
`retained_edges` checks nor the scan; a node target no edge type references,
an edge overwrite, an append and a merge carry `none` as well. Under `none`
no removed id is found, so the removed-id allowance of `write_max_bytes`
charges nothing. Retained-edge access describes the filtered
lookup in the incident edge table. Append and merge keep the consulted table,
so their `edge_ri` and non-key `@unique` checks use `probe` only when every
looked-up column carries a BTREE covering every fragment of that table (the
`@key` column's own index, or an `@index` on an orderable scalar; an `@index`
on a String is a full-text index and does not count) and `scan` otherwise, a
partly covered index included: an unindexed probe hands the store one filter
of every key, copied per fragment for the whole read.
The `FragmentWrite` row's `mode` pairs with its `adapter`: an overwrite writes
`fragments`, an append uses `strict_insert`, and a merge uses `upsert`.
The `plan` rows are the query explain's, with these values for a load:
| `node` | `detail` |
| ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `datafusion` | `unavailable`: a load's operators are lowered at execution, under the statement's transaction |
| `route` | `engine` for overwrite, which runs the plan; `executor` for append and merge, which the plan describes while the keyed adapters run them (`reason` then says `mode_on_executor`) |
| `explain_version` | the shared counter, `8` since a write's plan has the `DeltaSource` leaf |
| `operation` | `kind` (`load`), `mode`, and `tables`: the target and each consulted committed table with its `type_key` and `version` |
## A staged write's document
Every other write the engine validates is planned the same way, one plan
per table it stages: a JSONL load (`POST /load`, `omnigraph load`), a
`load csv` append or merge (whose one plan is the statement's own, above),
a GQ `insert`, `update` or `delete`, a Blob cell write and a `branch
merge`. The write's receipt carries one document per table it validated,
and the engine's judge executes exactly the `Validate` node each document
shows while the keyed adapters write. Such a document has two nodes:
| `node` | `detail` fields |
| ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Validate` | as above: `table`, `checks` with each check's `access`, `removal` |
| `DeltaSource` | `table`, `version` (the committed version the write pins, or `null` for a table new in the base), `rows` (the staged rows, exact), `delta` and `deletes` |
`delta` is what the write does to the table's rows: `insert` (a strict
insert, an append), `upsert` (a keyed upsert, a merge load, a staged
mutation, a branch merge's effects), `replace` (an overwrite's whole new
image) or `removal` (no rows: the write only removes ids). `deletes` says
whether the write removes ids from the table (a `delete`, a branch merge's
deletions); a removed node may strand committed edges, so a node table with
`deletes` carries the `retained_edges` checks an overwrite carries, and an
edge table with `deletes` under a non-zero `@card` minimum carries the
`cardinality` recount of the sources it empties.
The `plan` rows differ from a `load csv` statement's in two values:
| `node` | `detail` |
| ----------- | ------------------------------------------------------------------------------------------------------- |
| `route` | `executor`, always; `reason` says `write_on_executor` with the write's kind |
| `operation` | `kind` (`write`), `write` (`load`, `mutation`, `blob_cell` or `branch_merge`), `committed` and `tables` |
`committed` is the committed image the checks read: `base`, the
transaction's pinned base (a load); `live`, the base for existence and
uniqueness with `@card` counted on the live branch (a mutation and a Blob
cell write); `merge_target`, the merge target's pinned base (a branch
merge). The access rule is the one above: a kept table's `edge_ri` and
non-key `@unique` lookups `probe` only under full BTREE coverage of the
consulted table and `scan` otherwise, so a mutation inserting edges into a
Person table whose key index does not cover every fragment scans the
table's id column once, and probes it again once the table is optimized.
An `edge_ri` check's `probe` follows the same rule as a load's: under
`swept` the judge loads the committed key sets, one per endpoint node type,
before it sweeps the staged rows, and under `judged` it looks the endpoints
up after the sweep. A load and a mutation refuse on the first missing
endpoint; a branch merge lists every one among its conflicts.
A GQT case pins these documents with `--- expect plan` after a mutate
step's expect or a `branch merge` step's expect, in the `load validate …`
lines of a load plan.
An executed load's receipt carries the same document and an `executed`
object with `partitions` (the widest operator's), `rows`, `source_bytes` and
`operators`: one entry per node of the plan in post-order (the source
first), each with the node's `id` and `node` kind, the `operator` it built,
the `rows` it emitted (the sink counts the rows it staged),
`elapsed_compute_ns`, the time its stream and its workers were polled,
`wall_ns`, the span from its first poll to its end with every wait included,
and `partitions`, the streams it ran (1 for every operator but the source).
The source's entry also lists `slices`, one per record-aligned 8 MiB slice of
the source in source order: `slice`, the `partition` whose byte range held it,
its `rows`, `read_ns`, the wait for the slice's bytes, `elapsed_compute_ns`, and
`start_ns` and `end_ns` from the source's first poll. The statement's answer, the `ChangeOutput` of
`omnigraph mutate --json` and of `POST /mutate`, carries the object as
`executed` beside the counts and the commit. The compute, wait and wall numbers
are measurements of one run; a GQT case pins only the rows and the partitions.
# Explain
Source: https://www.omnigraph.dev/docs/queries-search/explain
`explain` before a read declaration returns the query's v2 plan instead of
its rows, regardless of the selected execution engine; before a `load csv`
statement it returns the load's plan ([explaining a load statement](https://www.omnigraph.dev/docs/queries-search/explain-load.md)).
It is a statement:
one per file, never beside a declaration. The query is compiled and planned
against the target's schema and snapshot without executing it. Planning
may read metadata for byte estimates and index planning, including row-ID
and deletion metadata for overlays or index-detail discovery on legacy tables;
it does not execute query rows
or invoke the embedding client.
```graphql
explain query adults() {
match {
$p: Person
$p.age > 30
}
return { $p.name }
}
```
```bash
omnigraph query --query explain.gq --target ./graph.omni --format kv
```
The result is a table with one row per plan node, in the order an `EXPLAIN`
listing prints them:
| Column | Type | Meaning |
| -------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `tree` | `String` | `logical_initial`, `logical`, `physical` or `datafusion` for a plan node; `plan` for the rows after the trees |
| `depth` | `I64` | the node's depth from its tree's root; absent on `plan` rows |
| `node` | `String` | the node kind (`TableScan`, `Filter`, `Expand`, `Projection`, …) or the operator (`ScanExec`, `FilterExec`, …); on `plan` rows the field name (`pass`, `route`, `logical_hash`, …) |
| `detail` | `String` | the node's own fields as JSON, without its children; an operator's own text; on `plan` rows the field's value, a string bare and anything else as JSON |
Every fresh physical `Scan` records `access`. `sequential` disables scalar
indexing. `index_probe` includes `index` (sorted names), `index_query` (a
Boolean tree with index name, physical column and Lance-rendered search at
each leaf), and `residual` (the indexed branch's remaining predicate, or
null). Uncovered fragments still apply the full filter. `runtime` records a
`reason`: ranking, search membership, join-generated filters or a dynamic
expression can add inputs unavailable during planning. Those reads retain
Lance's default. A dependent `id_lookup` adds `index`, its usable identity
BTREE name or null. A saved plan predating these fields retains the default.
A bare equality on a single String key can gain an identity predicate when
its ID index is usable. The written equality remains. Other key types are
not narrowed because supported historical IDs may use different spellings.
The `logical_initial` tree comes first (the logical plan as resolved, before
rewrite passes ran; the document's `logical_plan_initial`), then the rewritten
logical tree, then the physical tree, then the `datafusion`
tree, each in pre-order, so a node's children are the rows one level deeper
that follow it and the tree is rebuilt from the rows without loss. Every
physical row's `detail` carries `id`, the node's integer id in the physical
plan: unique within one plan, not stable across plans, and absent on
`logical` and `datafusion` rows. The explain document
(`Session::explain_query`) carries the same `id` on every node of its
`physical_plan`. A physical
`Expand` row's `detail` carries `mode` (`indexed_scan` or `csr`, the
traversal path the planner chose), `alternatives` (the modes the run may
switch to: the other mode when the cost model chose and may re-decide from
the observed frontier or a warm CSR; empty when
the mode was pinned or chosen without statistics), `frontier_estimate`
(the row-count estimate it chose from, `null` when the snapshot holds no
statistics). For budgeted expansion, this is the input row-count estimate.
It records `edges` as an object with `kind` (`named`, `alternation`, or `wildcard`)
and `members`, an array of `{ "edge_type": "Knows", "direction": "out" }`
objects. Directions are `out`, `in`, or `both`; names are canonical schema names.
`src_type` and `dst_type` are the endpoint types. `versions` maps every member name to its pinned dataset version or `null`
when absent. Budgeted selections have no CSR alternative. A traversal's destination is reached one of two ways. A
physical `Scan` row with `id_restriction` `input` reads the destination once
per slice of at most 256 traversal rows and carries `access` `id_lookup`. A
physical `HashJoin` row reads the destination table once (its second input,
a table `Scan` of it) and joins the traversal (its first input) to it; it
carries `binding`, the joined binding, and `fallback`, the access path the
run may take instead: `id_lookup` when the build refuses memory before the
traversal is consumed, `null` when the plan declares none. A run takes no
switch the plan does not declare; which side ran is in the
[`profile`](#profile) rows, never in explain. A physical `Scan` row of a
binding a leading `nearest()` or `bm25()` ranks carries `ranked`: an object
with `kind` (`nearest` or `bm25`), `property`, `query` (the argument as the
query wrote it, a parameter name such as `$q` or a literal, never its bound
value), `fetch` (the candidates the index is asked for: the query's limit
for `nearest`, `null` for `bm25`, which is uncapped), `nprobes` on a
`nearest` scan (the `ann_nprobes` setting when the plan was built, the IVF
partitions probed per index delta; `null` is no cap) and `scope` (`order`
for the query's own order, `primary` or `secondary` for an arm of `rrf()`).
A `nearest` scan's `policy` declares what the run may do when an attempt
leaves it short: `probe_factor` (each rescan multiplies the probe cap),
`flat_rescan_on_unreached`, `uncapped_on_missing_counters` and
`flat_when_eligible_within_fetch`. A `bm25` scan's `eligibility` says where it
applies its filter: `before_scoring` when the property's full-text index
covers every fragment of the pinned version, `after_scoring` otherwise, so a
filter never changes a BM25 score. A
physical
`RankFuse` row has two inputs, one subtree per arm, each with its own ranked
`Scan`; it carries `arms` (binding and kind per arm), `k`, `limit` (the
query's, which sizes the nearest arm's candidate window; the fused rows are
cut by the `Limit` above the `Sort`) and `row_tiebreak`. A physical `AntiJoin`
row (a correlated block: `not`, `exists`, `count` and the column aggregates)
carries `strategy`: `correlated` when its second input starts from the outer
rows through an `OuterReference`, `key_set` when the block's pattern runs once
from its own most bounded binding (a key or unique equality or a list
membership) and ends on the correlation binding, whose identity keeps or drops
each outer row. The planner chooses `key_set` only for a block that reads
nothing of the outer rows but that identity, and only when its row estimate is
lower; it then records the pass `anti_join_strategy` and the statistic
`anti_join_rows($x)` with both estimates. Either strategy returns the same
rows. These downstream
row keys follow the arm's score and fused node ID; selected edges use
`$e.@type` before `$e.@id`. Fusion still scores the ranked node by its ID and
appends the fused score as `._rrf`. A
physical `CrossJoin` row carries `filters`, the conjuncts over both bindings
it keeps pairs by, when it has any. A physical `ContainsJoin` row is the
planner's join for one `$r.x contains $l.y` conjunct: it carries `haystack`
(`$r.x`, the text searched) and `needle` (`$l.y`, the text searched for) as
GQ spells them, and `residual`, the other conjuncts of the same filter as
text; its second input is the table `Scan` of `$r`, whose `runtime_filter`
object names the `column` the join filters at run time (`x`), the `needle`
as `[binding, property]` and the `kind` (`text_contains_any`). A `Scan` row
carries no `runtime_filter` key unless a `ContainsJoin` marked it. A physical
`HydrateColumns` row at the root of a query plan (pass
`late_materialization`) carries `bindings`: per binding its `binding`,
`table` and the `columns` the output alone reads, which the scans below no
longer project (they read the row address instead) and which are fetched by
row address for the rows the root `limit` kept. Its `properties` carry
`retained_limit`, hydration's chunk budget: a quarter for what the storage
layer may read ahead, the rest for one chunk's fetched rows and their copies.
It is a budget, not a cap on all memory: the row that crosses it, up to eight
decoded rows waiting to be charged and the storage layer's soft read-ahead,
which can pass its share by one coalesced read, all come on top, as
[execution.md](https://github.com/ModernRelay/omnigraph/blob/main/docs/dev/execution.md#v2-plan-lowering-one-node-one-operator)
states. Then one
`plan` row per optimizer pass that fired (`node` `pass`), and one per
remaining field of the explain document: `route` (the engine route the plan
describes), `logical_hash` (the structural hash of the logical plan),
`explain_version`, `operation`, and the statistics used by planning; then
`validation`, what plan acceptance checked: `{"scope": "exact_subset"}` when
the query is a member of the exact fragment (one binding, Boolean, integer and
String filters, an optional leading `bm25()`, property sort keys, a limit and a
projection of properties and the score) and every rewrite from its canonical
form was checked, `{"scope": "invariants_only"}` when the plan was checked
against the query's requirements alone. A plan that fails acceptance is never
explained: the query fails as it would fail to run. The
last `plan` row is `assumptions`, what the planner read to build the plan,
as one JSON object: `params`, the names of the parameters it read (never a
value); `settings`, each setting it read with its spelling (`traversal`,
`ann_nprobes`); `env`, each environment variable it read with the value it
resolved (`OMNIGRAPH_EXPAND_INDEXED_MAX_FRONTIER`,
`OMNIGRAPH_EXPAND_INDEXED_MAX_HOPS`, when the query traverses); `gate_policy`,
the `rrf_plan` mode and the prefilter gate's admission thresholds;
`full_text`, the full-text coverage of every property a `bm25` scan ranks,
keyed `
.` (`full` only when the index is known to cover every
fragment, `partial` when coverage is incomplete or unknown, `absent` without a
full-text index); and
`memory_limit`, the query pool in bytes. Selector statements also capture
`traversal_work_limit`, shared across the execution, and `has_wildcard_traversal`,
retained through rewrites so historical replay can enforce the target rule. Every
planned document carries `pipelines`, the physical plan cut at each node whose
operator emits nothing before it has seen its whole input (`Sort`, `Aggregate`,
`AntiJoin`, `RankFuse`, a load's `Validate`, a costed multi-hop `Expand`) and
at a join's build side (a hash join's build, the collected left side of a
cross join or a contains join): each entry names its `source`, its
`partitions` (the streams its source runs: a `load csv` source's
`max_partitions`, 1 for every other source), its streaming `operators` in run
order, the `sink` it drains into (the breaker node, a join's build side, or
the `consumer`) and `depends_on`, the pipelines drained before it starts. The pipelines describe the plan: a correlated block the
engine answers from the bulk CSR degree (a single-hop, filter-free existence
or row-count test) prints its inner pipeline without running it. Query plans
omit node `properties.schema`: complete pipeline
schemas are derived when lowering. Result nodes declare their output through `columns`. Every query scan records its pinned `version`, or
`null` when that type has no dataset in the requested snapshot.
The explain document's `statistics` includes one `index_facts(
)`
entry per read table, with index names, columns, kinds and fragment coverage.
Its origin identifies the pinned dataset version. Unknown coverage is `null`.
Traversal costing uses these facts; absent, unusable or incomplete endpoint
indexes receive scan cost. Saved plans retain this coverage. Older accepted
plans without coverage provenance continue to check coverage during execution.
Sort keys and ordering use GQ text such as `$p.name desc`; a ranked scan's
ordering names `$p._distance asc` or `$p._score desc`, and the physical
`Sort` above a search order leads its `keys` with that score key, followed by
the query's plain keys; a fusion's ordering is `rrf($a, $b) desc` and it
plans no `Sort`. A `Sort` row's `tiebreak` lists the metadata keys it appends after
`keys` (`$p.@id`, and `$e.@type` plus `$e.@id` for selected bound edges, the
bindings the query declares in name order, then any binding the query does not
name), empty where equal rows are indistinguishable; its `ordering` property
is its whole comparator, those keys included as `$p.@id asc`, and a `Filter`
keeps its input's ordering.
The `datafusion` tree is the plan the query executes on the `v2` route: the
physical tree lowered to operators, every read operator omnigraph's own
(`ScanExec`, `ExpandExec`, `HashJoinExec`, `FilterExec`, `ProjectionExec`,
`SortExec`, `LimitExec`, `HydrateExec`, `CrossJoinExec`, `ContainsJoinExec`,
`AntiJoinMaskExec`, `RankFuseExec`,
`MetadataCountExec`) except the aggregate, DataFusion's `AggregateExec`, one
row per operator with `depth` from
its nesting and `detail` the operator's own text (a filter's predicate, a
sort's keys, a scan's columns; `AntiJoinMaskExec` serves every correlated
block, `not`, `exists`, `count` and the column aggregates, and prints its
predicate, then `key_set` under that strategy). It is the tree of the query's first pass; a
search that widens its scan after a short answer lowers it again. An
`explain` runs nothing, so when the lowering would need the embedding client
(a `nearest()` whose query is a string), the tree is left out and one `plan`
row `datafusion` reads `unavailable: …` with the reason.
## Profile
The run's `profile` describes each plan node in the explain row schema
(`tree`, `depth`, `node`, `detail`). Rows use `tree` `profile`, no `depth`,
the plan node's kind in `node` (`Scan`, `HashJoin`, `ContainsJoin`, …),
and its operator's JSON report in `detail`:
| Key | Meaning |
| ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `id` | the node's `id`, the same integer the physical explain row carries |
| `operator` | the operator's name (`ScanExec`, `HashJoinExec`, `SortExec`, …) |
| `status` | `executed` (a stream of the operator was polled) or `skipped` (in the tree, never polled) |
| `attempts` | one entry per pass: `rung` (0 the first pass, then the overfetch rung taken), `ran`, `actual_rows` (the operator's output rows) and `drained` (whether `actual_rows` is complete, defined in [execution.md](https://github.com/ModernRelay/omnigraph/blob/main/docs/dev/execution.md#v2-plan-lowering-one-node-one-operator)) |
`ran` is `true` or `false` on an operator without a choice, and on the
operator of a declared switch it is the side that ran: `hash_join` or
`id_lookup` on the `HashJoinExec` of a `HashJoin`, `csr` or `indexed_scan`
on the `ExpandExec` of an `Expand`, the mode the traversal ended on. After
the node rows, one row per search decision the run took within its plan's
declared policy: `node` `gate` (a pre-pass gate's `plan`, `prefilter`,
`postfilter` or `proven_empty`, with its `fallback`, `forced`, `eligible` and
`corpus`) or `probes` (a `nearest` scan's `attempts`, each with its
`maximum_nprobes`, `flat` and `rows`), each with the node's `id` and the
overfetch `rung`. Explain itself runs nothing and carries no `profile` row; the
profile is returned beside the rows by the run that produced them
(`Session::query_inspected`, the v2 inspection door, through
`Executed::profile`). The row schema is `explain_version` 8 (version 8 added
the `DeltaSource` node a staged write's plan starts from): `pipelines`
entries (on every planner document since version 7, on diff and merge
documents only before it) carry a pipeline `id` and `depends_on`, with
`source`, `operators` and `sink` as objects whose `id` joins the
`physical_plan` node ids; an operator's emission is set by the shape wrapper
the engine builds its stream through, the one the planner's cut names, so the
printed pipelines are the executed ones. Saved physical
plans use a `bound_plan_version: 11` envelope whose `body` contains `plan` and
`values`. Unversioned plans and other versions are refused with a regeneration
instruction, including plans without traversal nodes.
A nearest scan also reports `node` `ann_metrics` for each measured attempt:
| `detail` key | Meaning |
| ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `id`, `rung`, `attempt` | the scan node, overfetch pass and zero-based position in that pass's `probes.attempts` list |
| `counts` | available `partitions_searched`, `partitions_ranked`, and prefilter loads, input rows, input batches and row IDs |
| `cumulative_times_ns` | available index-open, partition-load/preparation, prefilter-wait, CPU-queue-wait, search, query-preparation, distance/top-k and result-materialization timings, plus prefilter load/input/build timings, in nanoseconds |
`partitions_ranked` sums the total IVF partitions in each opened index delta;
`partitions_searched` counts the partitions actually searched. Their ratio is
not a recall measurement. This is a curated subset of native diagnostics;
index cache hits, misses and comparison counts are not exposed here.
Missing metrics are absent, not zero; an attempt without metrics has no row.
Timings include waits, overlap and accumulate across concurrent partitions;
their sum is not query latency. Diagnostics may vary on replay and do not
change probe decisions, result ordering or replay policy.
An operator that wrote scratch files reports `node` `spill`, one row per
pass that spilled:
| `detail` key | Meaning |
| --------------- | ------------------------------------------------------------------------------------------------------------------------- |
| `id`, `rung` | the plan node and overfetch pass |
| `spill_count` | the spill files the node's operator wrote, including a sort it ran itself, such as the pair sort of a bound-edge `Expand` |
| `spilled_bytes` | the bytes of those files as written; spill files are zstd-compressed, and the query's scratch quota counts these bytes |
| `spilled_rows` | the rows written to them |
A pass with no spill has no row. Whether an operator spills depends on the
query's memory budget, so these rows can differ between runs of one plan.
An `Aggregate` node's `aggregates` array aligns with `exprs`, with `null` for
group keys. Each aggregate records its result `column`, `func`, declared `input`
and `result` types, `accumulator`, and `overflow` conversion. A type's `?` marks
declared nullability. Integer `sum` uses `exact_integer` and `round_to_nearest`;
floating-point `sum` and `avg` use `float64`.
Aggregate, Projection and MetadataCount also print `columns`, an ordered list
such as `["total: F64?", "person: Person"]`. Names match executed result columns;
`?` records declared nullability, so `count` is `I64?` even when execution
returns no nulls. A node type names its complete projected object, excluding
Blob and Vector properties.
Expression-bearing plan fields have additive `typed_*` counterparts. Each
expression tree records `op`, the existing GQ text in `gq`, its stored `type`,
and ordered child trees in `args`. For example, `typed_exprs`, `typed_filters`,
`typed_filter`, `typed_keys`, `typed_left`, `typed_right`, `typed_residual` and
`typed_k` cover results, filters, sorts, block operands, join residuals and
rank constants. Ranked scans also expose `typed_query` and `typed_score`;
ContainsJoin exposes its retained `typed_conjunct`. A scan's `typed_filter`
contains its GQ conjunct trees; storage-only restrictions have no GQ tree.
An AntiJoin exposes its complete left block tree in `typed_left` and bound in
`typed_right`. Its `aggregate` records the aggregate leaf's `gq`, `func`, input
and result types, accumulator and overflow rule; bare row count has `null`.
Comparison casts wrap the finalized aggregate result. Integer block sum
therefore accumulates exactly, rounds once to F64, and then compares in the
stored domain. Row count is I64 non-null; column aggregates remain nullable.
A `cast` tree stores the converted type and one child carrying the source
type. Its `gq` text is identical to that child's, so existing text keys remain
stable. Types come from compiler declarations, including `?` nullability;
explain does not infer them. Mixed signed and U64 comparisons can use internal
`exact_integer` values (Decimal128), including list elements. This internal
type never appears as a public result column.
An `explain` statement is served by `omnigraph query` and `POST /query`. It
takes the same `--branch`/`--snapshot` target and `--params` as the query
itself and needs the same `read` policy decision. `mutate` refuses it, as does a
mutation declaration under `explain`. A load statement under `explain` takes
the same doors with a `--branch` target only: a historical `--snapshot` is
refused, since a load targets a branch head. Served load explain requests
require the load's `change` permission on that branch before source access.
Every read executes on engine v2, the one value of the session setting
`engine`, so explain describes the route the query runs. See
[session settings](https://www.omnigraph.dev/docs/queries-search/queries.md#session-settings).
# Mutations and Loads
Source: https://www.omnigraph.dev/docs/queries-search/mutations
Mutation statements live inside a named `.gq` query. Run them with
`omnigraph mutate`.
```graphql
query hire($person_id: String, $company_id: String, $role: String) {
insert WorksAt {
from: $person_id,
to: $company_id,
role: $role
}
}
```
Edge endpoints use the reserved assignments `from` and `to`; their values are
the logical ids of existing endpoint nodes.
## Statements
```graphql
insert Person { email: $email, display_name: $name }
update Person set { display_name: $name } where email = $email
delete Person where email = $email
```
A `where` on `update` or `delete` takes the same expressions as a read filter
(see [Boolean expressions and nulls](https://www.omnigraph.dev/docs/queries-search/queries.md#boolean-expressions-and-nulls)):
`and`, `or`, `not`, parentheses, the six comparisons, `starts_with`,
`contains`, `in`, `is null` and `is not null`. Its operands are the target type's
properties, named bare, the system fields `@id`, `@src` and `@dst`, literals,
parameters, and `now()`; `from` and `to` stay accepted for `@src` and `@dst`.
It keeps only the rows whose expression is true. On four `Knows` edges, `ab1`
and `ab2` from `a` to `b`, `ac`, and `db`:
```graphql
query exact() { delete Knows where @src = "a" and @dst = "b" }
```
removes `ab1` and `ab2` and nothing else: `affected: nodes=0 edges=2`. A binding
variable in a `where` is refused at compile time (`T14`); an aggregate, search,
or ranking call is `T44`, for example ``T44: `count` cannot appear in a mutation
where; a where compares the row's own properties, parameters and now()``. A
compound `where` runs under either `engine` setting. A `where` that tests over
four properties twice each with `=`, `!=` or `in` is refused.
Assignment values are constants: literals, parameters, `now()`, and comparisons
or Boolean operators over them, such as `adult: true or $flag`. A constant's
value is fixed per invocation: parameters and `now()` are bound once before any
retry, so a retried mutation computes the same value. It follows the read rules
for null: with `$flag` null, `true or $flag` is `true` and `false and $flag` is
`false`, and `$age > 30` with `$age` null is null. A null result assigned to a
nullable property writes null; on a `Blob` property it clears the cell. `null`
is a reserved word, so clear a Blob through a nullable parameter such as
`$content: Blob?`. Assigned to a non-nullable property a null result is refused
with a typed error that names the property, never written as a default. A
property or system field in a value is refused, for example ``T45: `age` cannot
appear in an assignment value; assignments and binding matches are constants per
invocation``.
A mutation query may contain several inserts and updates, or several deletes,
but it cannot mix inserts or updates with deletes. Split that workflow into two
queries, or run the queries on a branch and merge when the combined result is
ready.
## Atomicity
An effectful mutation query publishes one graph commit. All of its statements
become visible together, or none do. Separate mutation commands are separate
commits, even when they run consecutively.
With `--json`, a successful effectful mutation returns `commit` with the
exact `graph_commit_id` and commit metadata published by that attempt. A
successful mutation that matches no entities publishes no commit and returns
`"commit": null`. Load JSON responses use the same exact receipt.
For a multi-command workflow, use a branch as isolated staging. Earlier changes
remain committed on that branch; merging makes the resulting branch state
visible on the target in one atomic step. See
[Branches, Commits, and History](https://www.omnigraph.dev/docs/branching.md).
## Insert and update identity
* Inserting a node or edge with `@key` is an upsert by its derived id.
Inserting the same key again updates the existing row.
* Inserting a node without a key is a strict insert with a generated or supplied
id.
* Inserting an edge without a key is a strict insert with a generated or
supplied id; the same pair inserted twice is two edges.
* Key properties cannot be changed by an update. Edge types do not support
update at all: change a keyed edge by inserting its key again with the new
values (an upsert), and an unkeyed edge by delete and re-insert.
All declared value, uniqueness, endpoint, and cardinality constraints are
checked before publication; a node a `@card` minimum needs edges for arrives with them in one query or load ([edge cardinality](https://www.omnigraph.dev/docs/schema.md#edge-cardinality)).
## Bulk loading
`omnigraph load` accepts newline-delimited JSON. One file can contain nodes and
edges of several types:
```jsonl
{"type":"Person","data":{"email":"ada@example.com","display_name":"Ada"}}
{"type":"Company","data":{"slug":"acme","name":"Acme"}}
{"edge":"WorksAt","from":"ada@example.com","to":"acme","data":{"role":"Engineer"}}
```
Here, `Person.email` and `Company.slug` are single-property String keys, so their
derived ids are exactly `ada@example.com` and `acme`. The edge uses those ids in
`from` and `to`.
A `Date` value is an integer day count since 1970-01-01 or a `YYYY-MM-DD`
string; a `DateTime` value is an integer millisecond count since the Unix epoch
or an ISO 8601 string. Any other JSON type (a float such as `19723.0`, a
boolean, an object) fails the load with `invalid Date value` or `invalid
DateTime value` naming the property. A `Date` string carries no time of day:
`"2024-01-01T02:00:00+05:00"` fails the load with `invalid Date literal`, as it
does in a mutation param, a `date(...)` literal, or a read filter; an instant
belongs in a `DateTime` property. A `DateTime` holds milliseconds: a string with
a non-zero digit past the third fractional digit, such as
`"2024-01-01T00:00:00.123456Z"`, fails the load with `invalid DateTime literal`,
as it does in a mutation param, a `datetime(...)` literal, or a read filter.
Trailing zeros, as in `.123000`, are accepted.
Choose the mode explicitly:
| Mode | Existing id | Use |
| ----------- | ------------------------------------------------------------------------------------------------------- | ---------------------------------------- |
| `append` | Fails with `key_conflict` | Add entities without replacing anything. |
| `merge` | Updates the existing entity | Upsert a batch. |
| `overwrite` | Replaces every node or edge type represented in the batch; types absent from the batch remain unchanged | Rebuild from a complete export or seed. |
For a keyed edge the existing-id column applies to its derived id: `append`
reports `key_conflict` on an already-committed pair, `merge` upserts it, and
a supplied top-level `id` must equal the derivation exactly.
```bash
omnigraph load --data batch.jsonl --mode merge --target ./graph.omni
```
One load request is one graph commit. Use `--branch --from ` to
create a missing review branch and load onto it in the same workflow.
Loads preserve supplied embeddings and do not generate them. Results report
`embedding_generation: "unsupported"` when a loaded node type declares
`@embed`, or `null` otherwise; see [Embeddings](https://www.omnigraph.dev/docs/queries-search/embeddings.md).
### CSV loading
The `load csv` statement reads one CSV object into one declared node or edge
type, with every cell typed by the `.pg` declaration. It is a GQ statement, so
it runs wherever a `branch create` statement runs: `omnigraph mutate -e`, a
`.gq` file, or `POST /mutate` with the request `branch` as its target (`main`
when omitted). It is the one CSV surface: `omnigraph load` reads JSONL only.
```graphql
load csv "s3://imports/person.csv" into node:Person mode append delimiter "|" header
load csv "s3://imports/knows.csv" into edge:Knows mode append delimiter "|" header columns (from, to, creationDate)
```
* `mode` is required: `overwrite`, `append`, or `merge`, with the meanings of
the table above.
* `delimiter ""` is one printable ASCII character; `,` when unspelled.
* `header` makes the first record name the columns: each name is the property
it feeds, and an edge file's first two columns are its endpoints whatever
the header calls them (`Person.id|Person.id` in an LDBC file). In mode
`overwrite` the first record is read from the source's first 8 MiB, with or
without `header`; a first record longer than that is refused.
* `columns (a, b, …)` names the property each column feeds, by position. It
is required without `header` and overrides a header's names. `from` and
`to` name an edge file's endpoint columns and are refused on a node file.
* An empty field and `""` are null (there is no empty-string spelling); an empty line is not a
record, so a one-column file skips its null rows. A null for a non-nullable property, a column
the type does not declare, a count of `columns` that differs from the source, a malformed
number, a repeated name, and an edge file without `from` or `to` are refused before anything is staged.
Numbers are typed by the reader. `Bool` takes exactly `true` or `false`.
`Date` and `DateTime` cells are read as text and pass the checks above
before conversion: `2024-01-01T02:00:00+05:00` in a `Date` column and
`2024-01-01T00:00:00.123456Z` in a `DateTime` column fail the load exactly
as they fail a JSONL load, and a numeric offset without a colon, as LDBC
writes it (`2010-02-14T15:32:10.447+0000`), stores the instant. List, Vector
and Blob properties have no CSV encoding: a column mapped to one is refused
with the kind named; an unmapped one stores null when nullable.
Float values must be finite and representable by their declared `F32` or
`F64` type; NaN, infinity and overflow refuse the load. Each decoded batch
is held against the statement's `query_memory_bytes` allowance before it
is retained, and each staged-write input against `write_max_bytes`.
Node ids derive from the `@key` as in a JSONL load; an edge's `from` and `to`
are the canonical ids of its endpoints, so a single-property integer key is
written as its decimal value. The statement never forks a branch.
The source is a URI the engine reads under the graph's external Blob policy
([External Blob references](https://www.omnigraph.dev/docs/clusters/config.md#external-blob-references)):
`s3://` objects beneath a configured
base, and for an embedded run `file://` paths beneath an `embedded_only`
base. A graph with no policy denies every source; a URI outside every base,
an unsupported scheme, and an object over the
[`load_source_max_bytes`](https://www.omnigraph.dev/docs/queries-search/settings.md) setting (default 32 MiB)
are refused before any byte is read, and every read of the object (each
range an overwrite's partition reads, or the one read of an append or merge)
is bounded by a 120-second deadline. A refused source names `csv_source_bytes`; raising the setting in
the statement's `set` prefix (or the request's `settings` field) admits a
larger object. A direct `omnigraph mutate`
run installs an embedded-only base over a `file://` source's directory for
that statement, so the file is admitted by the same rule; a served run
refuses a `file://` source, since the server reads only what it may reach.
`explain` before a `load csv` statement prints the load's plan, the same
document a query's explain prints: the source, the one validation with each
check and its access path, and the write.
An overwrite runs that plan; append and merge run the keyed adapters.
It reads the source's metadata and no payload byte.
See [Explaining a load statement](https://www.omnigraph.dev/docs/queries-search/explain-load.md).
#### CLI and HTTP requests
```bash
omnigraph mutate --target ./graph.omni -e 'load csv "file:///data/person_0_0.csv" into node:Person mode append delimiter "|" header'
omnigraph mutate --target dev -e 'load csv "s3://imports/knows.csv" into edge:Knows mode append delimiter "|" header'
```
The CLI uses `--branch`; HTTP uses the request `branch`, with `main` as the
omitted default. The statement accepts no `name`, `params` or commit
precondition (`--if-commit` or `Omnigraph-If-Graph-Commit`). It requires
`change` on the branch. A source that holds no records is refused. The
response is a mutation's `ChangeOutput`: the
branch, `load csv` as `query_name`, loaded node and edge counts, the commit,
and for `mode overwrite` an `executed` object, what the load's plan measured
as it ran ([explaining a load statement](https://www.omnigraph.dev/docs/queries-search/explain-load.md)).
```bash
curl -sS http://localhost:8080/graphs/knowledge/mutate \
-H 'Omnigraph-Http-Api: 0.14' \
-H 'authorization: Bearer secret-a' \
-H 'content-type: application/json' \
-d '{"query":"load csv \"s3://imports/person.csv\" into node:Person mode append delimiter \"|\" header","branch":"main"}'
```
The HTTP body carries the statement; the server reads its source under the
graph's `external_blobs.allow` policy, with server-safe bases only. Before
fetching, workload admission reserves the request's `load_source_max_bytes`
(default 32 MiB) plus the statement bytes against both global and per-actor
budgets, so a raised setting must still fit the server's own limits. This reservation lasts until the write
settles, including when the HTTP caller disconnects.
## Limits and conflicts
Insert/update mutations and incremental keyed loads are bounded to 8,192
entities per touched type. The [`write_max_bytes`](https://www.omnigraph.dev/docs/queries-search/settings.md)
setting (default 32 MiB) bounds retained row data per type and across all
touched types. Blob payloads have a separate allowance of the same size,
including inline bytes, copied external bytes and carried update values.
Every version of a key written in one mutation counts, so an insert and an
update of the same row charge its Blob twice.
Keyed loads also bound their parser estimate independently. A strict JSONL
load retains its projected input check, `strict_input_arrow_bytes`.
A vector counts its full width (dimension × 4 bytes), present or null: 12 KiB for `Vector(3072)`.
A CSV load has no whole-statement input check; it applies `write_max_bytes` to each staged-write input instead.
An overwrite stages the reader's 8,192-row batches, and a batch over the
allowance refuses as `staged_write_input_bytes`.
An overwrite writes a fragment per parse partition, up to 1 Mi rows each, as
the rows drain; nothing commits before the publish, a refusal deletes them all.
An append or merge keeps the keyed adapter's ceilings
(`retained keyed batch bytes per operation`, `keyed entities for
`,
`keyed entity bytes for
`).
The [`query_memory_bytes`](https://www.omnigraph.dev/docs/queries-search/settings.md) setting bounds one
`load csv` statement's tracked memory as a whole: the retained source, every
decoded batch, validation state and the staging working set; a streaming
overwrite's working set is its open fragments and the writers in flight.
An overwrite admits that memory once, before any partition reads, one share per
parse partition that only its own reader charges, so a refusal and its `actual`
(the smallest admitting limit) are the same on every run (the shares: the
settings reference); a statement it cannot hold refuses before any fragment write.
The source object is bounded by `load_source_max_bytes` before it is read
(refused as `csv_source_bytes`).
An overwrite probes the object's metadata once, then each parse partition reads
its own byte range in 8 MiB pieces, every read conditional on the etag and size
the probe captured; a refusal of the partitioned run reads the ranges once more
as one partition. An append or merge reads the object once per statement under
the same validators, and every reprepare attempt reuses those bytes.
An object replaced between the probe and the read refuses as an
external-source error, and nothing is staged.
Raising `write_max_bytes` lifts `strict_input_arrow_bytes` and
`staged_write_input_bytes`; raising `query_memory_bytes` admits a larger
statement; raising `load_source_max_bytes` lifts `csv_source_bytes`. No setting
lifts a merge's [validation limit](https://www.omnigraph.dev/docs/branching/merge.md#validation-limit),
which counts a branch's commits together. See [Blob limits](https://www.omnigraph.dev/docs/schema/blobs.md#limits).
Deletes, including cascades, and overwrite loads collecting replaced IDs have
a separate `write_max_bytes` allowance per operation for those IDs, summed over all
touched types. Each removed ID is charged its UTF-8 length plus 24 bytes, so
the default allowance holds 671,088 IDs of 26 bytes, the length of a generated ID. A
delete and the edges it cascades to draw on the same allowance. An overwrite of
entities loaded without a `@key` and without an explicit `id` removes every
committed ID of that type, because those IDs are generated again on each load.
An overwrite collects replaced IDs only when its plan's `removal` is `scan`, a
node target with committed rows that an edge type references; an edge target,
a node target no edge type references and an empty table collect none and draw
nothing from this allowance.
Oversized work returns a resource-limit error before its data is staged or
published. These checks do not bound total engine memory. The setting can
change each allowance from its 32 MiB default; a CSV overwrite stages its
input as a vector of reader batches, each bounded by the allowance, under
the statement's `query_memory_bytes`. Split larger inserts, updates, keyed
loads and deletes into explicit commits. An overwrite replaces each
represented type as one image and cannot be split: it keeps its bulk-input
behavior and remains subject to its removed-ID check.
Independent existing constructive datasets stage concurrently. The
`stage_write_concurrency` [session setting](https://www.omnigraph.dev/docs/queries-search/queries.md#session-settings)
controls that width for both Load and insert/update mutations (default 8,
range `1..=64`); it is `process` scope, so the server takes it from
`OMNIGRAPH_LOAD_CONCURRENCY` and a direct CLI run from that variable or from
`--set` on `load`, `ingest` or `mutate` (a JSONL input carries no `set` line),
and an invalid or `0` value refuses startup instead of running the default.
The same width is the number of Lance writers a `load csv` overwrite runs at
once: its batches are cut into runs of at most one Lance file (1 M rows), each
run written by its own writer, and the files are published in input order, so
the table's files are the ones a single writer would have produced. It is also
the most streams that parse the overwrite's source: each parse partition owns
an equal byte range of the source, reads it in 8 MiB pieces and cuts its
records on record ends, neighbouring partitions meet at one record end, and
the validation puts the slices back in source order, so the rows and their
order are those of a single parser. A boundary the neighbours cannot agree on
(inside a quoted field, or after an empty line) refuses that run, and the
same plan reruns as one partition. Delete staging remains serial. This affects
preparation only—one request still publishes exactly one graph commit.
A write overtaken before it takes effect, an update or delete included, reruns on the new head; it returns
`read_set_conflict` only if the branch was recreated, the schema changed, or another process kept overtaking
it. A `key_conflict` means an append or strict insert found an existing id; it never silently becomes an upsert.
If a write returns `recovery_required`, do not immediately resubmit it. Reopen
the graph read-write or restart the server, then retry from a fresh branch head.
## Blobs
Blob assignments accept managed `base64:` data and, when allowed by graph
policy, external URI references. Ownership differs by load mode, and Blob bytes
count toward write limits. See the canonical [Blob guide](https://www.omnigraph.dev/docs/schema/blobs.md) before
loading them, and its [limits](https://www.omnigraph.dev/docs/schema/blobs.md#limits) for the bounds that apply.
## Conditional mutations
A mutation can be made to apply only to the graph state the caller read, with
`--if-commit`; see [conditional mutations](https://www.omnigraph.dev/docs/queries-search/conditional-mutations.md).
# Query Language (`.gq`)
Source: https://www.omnigraph.dev/docs/queries-search/queries
A `.gq` file contains named, typed queries. Read queries match graph patterns
and return columns; mutation queries use the same declaration form and are
covered in [Mutations](https://www.omnigraph.dev/docs/queries-search/mutations.md). A file may instead hold exactly
one statement, never beside a query declaration: a branch statement (`branch
create`, `delete`, `merge`, `list`, `diff`, or `merge-base`; see [Branches,
Commits, and History](https://www.omnigraph.dev/docs/branching.md)), a `show` statement (see
[session settings](#session-settings)), or an `explain` statement, which
answers the plan a read query would run under instead of its rows (see
[Explain](https://www.omnigraph.dev/docs/queries-search/explain.md)). Any of them may open with
[session settings](#session-settings) lines.
```graphql
query engineers($title: String) @description("People with a title") {
match {
$p: Person { title: $title }
$p worksAt $c
}
return { $p.name, $c.name as company }
order { company asc, $p.name asc }
limit 50
}
```
Run an ad-hoc query with:
```bash
omnigraph query engineers --query queries.gq \
--params '{"title":"Engineer"}' --target ./graph.omni
```
## Declarations and parameters
```text
query ($required: String, $optional: I32?) { ... }
```
Parameter types use the schema scalar types. Every non-nullable declared
parameter must be supplied; omission fails before query execution. A trailing
`?` accepts `null` or an omitted value. `@description("...")` and
`@instruction("...")` attach metadata for clients that expose stored queries
as tools. An integer parameter carries the whole range of its declared type,
so a `U64` parameter takes every value from `0` to `18446744073709551615`.
## Match patterns
A match binds nodes and follows typed relationships. For edge alternatives,
wildcard, bounded hops, direction and edge identity, see
[Traversal and match patterns](https://www.omnigraph.dev/docs/queries-search/traversal.md). Omitted bounds always mean
exactly `{1,1}`; multiple hops require an explicit finite range.
### Boolean expressions and nulls
Filters combine with `and`, `or`, `not` and parentheses. `or` binds loosest,
then `and`, then `not`, then comparison, so `not $p.email is null` reads as
`not ($p.email is null)`. Comparisons do not chain: `$a < $b < $c` is a parse
error. `not {` stays the pattern negation. Operands of `and`, `or` and `not`
must be `Bool` (`T41`), as in
`($d.stage = "open" or $d.stage = "paused") and not $d.amount < $cut`.
A comparison (`contains` and `starts_with` included) with a null operand is
null, and `not null` is null. A filter or mutation `where` keeps only rows
whose expression is true, so `not ($p.age > 30)` skips rows whose `age` is
null.
Numeric comparisons choose their types before execution. A literal may use a
property's type when its value is exactly representable in that type; otherwise
both operands use a common numeric type. Parameters use their declared type,
regardless of the supplied value. An `F32` property storing `0.1` therefore
differs from the `F64` literal `0.1`, while `0.5` compares exactly. These rules
also apply to list membership and mutation predicates, whether a filter runs
in a scan or in memory. Mixed signed and `U64` values compare without losing
integer precision.
| `x` | `y` | `x and y` | `x or y` |
| ------- | ------- | --------- | -------- |
| `true` | null | null | `true` |
| `false` | null | `false` | null |
| null | null | null | null |
| `true` | `false` | `false` | `true` |
`x is null` and `x is not null` are always `Bool` and are the only null tests:
GQ has no `null` literal, so `$p.x = null` is a parse error.
`not ($p.age > 30) or $p.age is null` selects every row the comparison did
not. A parameter bound to `null` makes every comparison on it null. `is null`
refuses a `Blob`.
`and`, `or`, `not`, `is`, `null` and `in` are reserved words: none is a bare operand
or a return alias. `$p.and` after a dot and `and: 1` in an assignment or
binding match stay legal; `nothing` and `android` are ordinary identifiers.
A traversal names its edge bare or as a string, the one spelling for an edge
a reserved word names: `$i "in" $b` follows the edge `In`; `$i in $b` tests membership.
A mutation `where` takes the same expressions over the target type's
properties, `@id`, `@src`, `@dst`, literals, parameters and `now()`, never a
binding variable, aggregate or search call:
`delete Knows where @src = "a" and @dst = "b"`. Assignment values and binding
matches take constants evaluated once per invocation, such as
`adult: true or $flag`; a property or system field there is `T45`. See
[Mutations](https://www.omnigraph.dev/docs/queries-search/mutations.md).
### Correlated blocks
`not { ... }`, `exists { ... }`, `count { ... } op value` and
`sum(expr) { ... } op value` (also `min`, `max`, `avg`) match a pattern once per
outer row. Each block must read a variable bound outside it. The row is kept
when the aggregate comparison is true: `not` means `count = 0`, and `exists`
means `count > 0`. The right side is a literal, `now()` or typed parameter;
numeric bounds follow the [comparison rules](#boolean-expressions-and-nulls)
for the aggregate's result type. Its argument is a scalar expression in the
block's scope: numeric for `sum` and `avg`; numeric, `String`, `Bool`, `Date`
or `DateTime` for `min` and `max`, as in `return`. With no matches, `sum`,
`min`, `max` and `avg` satisfy no comparison; `count` is `0`.
Integer block `sum` accumulates exactly in 128 bits, refuses overflow of that
range, and rounds the total once to `F64` before comparison. For example,
`9007199254740993` and `-9007199254740992` sum to `1`, while a single value
`9007199254740993` rounds to `9007199254740992` and does not satisfy
`sum(...) { ... } > 9007199254740992`.
As in `return`, `avg` converts each non-null input to `F64` before summing
and dividing by the count. For those same two values, the average is `0`,
because the first value rounds before accumulation. `min` and `max` retain
the column's declared type, including the full `U64` range; comparing a `U64`
extremum with an `I64` parameter preserves both integer ranges. Row count and
column `count` use `I64` and refuse a count beyond its range; column `count`
skips null values.
The block narrows the rows before `order` and `limit`, so a paged listing
filtered by a relationship count is exact. A binding the block's traversal
connects to the outer row is reached through it, or the block runs once from a
node it binds by key if that is cheaper (`strategy` in [explain](https://www.omnigraph.dev/docs/queries-search/explain.md));
a binding correlated only through a filter, or read by a text search, is
scanned. A single-hop, filter-free `count { ... }` over a directed, unbound
edge is answered from the graph index's degree. A bare aggregate in `match`,
`count($d) > 2` without a block, is refused: it names no row to group by.
```
query prolific($least: I64) {
match {
$a: Author
count {
$p: Post { published: true }
$a authored $p
} >= $least
}
return { $a.name }
order { $a.name }
limit 20
}
```
### Strings and lists
* `$x.tags contains "rust"` tests membership when `tags` is a list.
* `$x.number in $numbers` tests membership in a list parameter or literal
(`$x.number in ["A-1", "A-2"]`): `$numbers contains $x.number` with the
list, of the value's type (`T7`), on the right. On a matched binding's
property it filters that binding's scan. An empty list has no member: `in`
keeps no row, `not (... in [])` keeps the rows whose value is not null.
* `$x.title contains "graph"` tests exact, case-sensitive substring containment
when `title` is a String.
* `$x.title starts_with "Omni"` tests an exact, case-sensitive prefix.
`NULL` never matches these predicates. `%` and `_` are ordinary characters,
not wildcards. The predicates remain correct without an index; do not assume a
free-text String index accelerates exact prefix or substring filters. Use
[full-text search](https://www.omnigraph.dev/docs/queries-search/search.md) for tokenization, fuzzy matching, and
relevance ranking.
### System fields
A node's identity and an edge's endpoints are system fields, read through
`@`-prefixed meta-fields that no user property can shadow: `$p.@id` is the
identity of any binding, `$e.@src` and `$e.@dst` the endpoints of a bound
edge. They work in filters, projections, and orderings (`$p.@id = $who`,
`return { $p.@id }`, `order { $p.@id asc }`) and answer on every graph,
whatever the stored column is called. In a mutation predicate, which carries
no binding, the bare form serves: `delete Person where @id = $who`,
`delete Knows where @src = $who`. A bare `id` (`$p.id`, `where id = ...`)
always names a user property called `id`; where none is declared it is the
unknown-property error, which names the meta-field. Edge inserts keep
addressing endpoints as `from` and `to`. `$e.@type` returns the bound edge's
canonical schema type name as a non-null `String`, including on ordinary named
traversals; it is synthesized query metadata and cannot be written. `@src` and
`@dst` preserve stored orientation, including when the query traverses in
reverse. In type position the same prefix names an engine-owned, read-only
type: `$o: @Operator` binds the rows of the [engine catalogue](https://www.omnigraph.dev/docs/queries-search/catalogue.md).
## Return, order, and limit
```graphql
return { $person.name, count($company) as companies }
order { companies desc, $person.name asc }
limit 20
```
Return expressions include variables, properties, literals, `now()`, and the
aggregates `count`, `sum`, `avg`, `min`, and `max`. Each gives one column named by
its alias or expression (`$p.name` gives `p.name`); duplicates are refused (`T25`).
Integer `sum` accumulates exactly in 128 bits and rounds once to `F64`; `avg`
uses a floating-point accumulator. `min` and `max` accept numeric, `String`,
`Bool`, `Date`, or `DateTime` columns and keep their type (false before true,
dates chronologically). With no matches, an aggregate-only return gives one row
(`count` 0, the rest null); a return that also projects a group value gives no rows.
A bare node returns one object with `@id` and every property except `Blob` and
`Vector`: `$p` produces column `p`; project a property for one field. `count($p)`
counts rows and `count(distinct $p)` distinct nodes; `count(distinct $p.city)`
counts distinct non-null values, so a node reached by several paths counts once.
Other aggregates require a property (`T8`) and refuse `distinct` (`T57`).
A comparison or Boolean expression requires an alias (`T43`), such as
`$p.age > 30 as adult`; its result is `Bool`, or `Bool?` with a nullable operand,
and `is null` is always `Bool`. In an aggregated return its operands are
aggregates, literals, `now()` and parameters, evaluated per group after
aggregation: `count($p) > 0 as any` is `Bool?` because `count` is `I64?`. A group
value beside an aggregate is refused (`T58`), any other Boolean item too (`T9`).
`return distinct { $p.name }` keeps one row per complete projected tuple.
Repeated traversal matches and equal values on different entities collapse;
nulls compare equal. Every projection is a key, including comparisons, and
bare nodes deduplicate by identity. Deduplication follows `match` and precedes
ordering and limiting, so the limit counts distinct rows. Aggregates with
`distinct` are refused (`T56`). Except for a leading search function, DISTINCT
order keys must be projected values or aliases, including property keys
(`T42`). Remaining projected columns break ties ascending; no identity tiebreak is added.
For ordinary returns, explicit order is total: entity ids break ties only when
they can change the visible order, and no id is read when every projection is a
key. Ascending puts nulls first, descending last. `nearest(...)` requires a limit.
On ungrouped, non-DISTINCT rows, property and system-field order keys need not be projected;
other keys, except the leading search function, must be return aliases or
expressions written in return (`T42`), like `count($d)` after `return { count($d) as deals }`.
For ordinary returns, `nearest(...)` orders by ascending distance and `bm25`
and `rrf` by descending score, including through traversals, ranking the binding named whichever binding `match` declares first ([details](https://www.omnigraph.dev/docs/queries-search/search.md#ranking-a-traversal-reached-binding); `rrf` arms on two connected bindings: `T59`).
The search function must lead; subsequent keys precede the identity tiebreak. Projected
scores must repeat that leading function (`T33`); the [Search guide](https://www.omnigraph.dev/docs/queries-search/search.md)
defines their names and types. Rank expressions under aggregates (`T32`) and
search predicates in return (`T35`) are refused.
For aggregate or DISTINCT returns, the leading search function selects the
input population; it does not order the output. Subsequent keys order output
rows, followed by every group or DISTINCT key ascending. Without subsequent
keys, output is unordered. Aggregates cannot project scores (`T9`). DISTINCT
uses the selected nearest population without retry overfetch, so duplicates
can leave fewer rows than the limit. BM25 reads every matching entity before
the final limit.
## Blobs
Blob properties are not ordinary read-query values. They cannot be projected,
filtered, ordered, or passed to an aggregate. Read one logical Blob cell with
the CLI or HTTP Blob endpoint described in [Blobs](https://www.omnigraph.dev/docs/schema/blobs.md). Blob parameters
remain valid for mutation assignment.
## Branches and historical reads
Reads default to `main`. Select another branch or an immutable commit with
`--branch` or `--snapshot`:
```bash
omnigraph query engineers --query queries.gq --branch review \
--params '{"title":"Engineer"}' --target ./graph.omni
```
When the snapshot has an effective graph head, `omnigraph query --json`
includes its `graph_commit_id`, pinned with the returned rows. Use that
same-snapshot id with a later mutation's `--if-commit` option when implementing
read-modify-write. On a newly created, unmodified branch, it is the head
inherited from the source branch and is valid for the branch's first
conditional mutation. See [Branches, Commits, and History](https://www.omnigraph.dev/docs/branching.md).
## JSON result spelling
JSON `rows` follow Arrow's JSON conventions: OmniGraph writes them with the
`arrow-json` writer from the result batches and keeps no per-type spelling of
its own. The spellings a consumer sees:
* A null cell's key is omitted from its row, and from a struct cell; a null
element inside a list value stays `null`.
* `Date` is `"2024-01-01"`; `DateTime` is `"2024-01-01T12:34:56.789"` in UTC
with no `Z`, and no fractional part when it is zero.
* Integers of every width are bare numbers; JavaScript's `JSON.parse` rounds
values beyond 2^53.
* `F32` prints at 32-bit width (`0.99`) and `F64` at 64-bit width; integral
floats carry `.0`; magnitudes from 1e10 up or below 1e-5 take exponent form
(`1.0e20`, `1.0e-7`); a non-finite computed value is `null`.
* `Vector(N)` and list properties are JSON arrays.
Send a 64-bit integer above 2^53 as a decimal string, so a client that parses
JSON through a double keeps every digit. On input, a JSON number for an `F64`
parses to the nearest `F64` value, the one a GQ literal with the same digits
names, so an `F64` value read from `rows` loads back unchanged. A `Date` string is a calendar day, `"2024-01-01"`; a
string that carries a time of day, such as `"2024-01-01T02:00:00+05:00"`, is
refused as a load value, a param, or a `date(...)` literal, and an instant
belongs in a `DateTime` property. A `DateTime` holds milliseconds: a string
with a non-zero digit past the third fractional digit, such as
`"2024-01-01T00:00:00.123456Z"`, is refused as a load value, a param, or a
`datetime(...)` literal; trailing zeros, as in `.123000`, are accepted.
A `Date` or `DateTime` count outside the range the writer can format is refused
on load. A read that meets one fails with status 500; the error names the
column, the result row, and the count, and an `update` of that row repairs it.
## Session settings
Use `set`, `reset`, and `show` to configure query execution. See
[Session settings](https://www.omnigraph.dev/docs/queries-search/settings.md) for syntax, scopes, defaults, and CLI/HTTP usage.
## Linting
Validate queries without running them; a refusal reports a stable code, its
position or stage, the expectation and one fix ([Diagnostics](https://www.omnigraph.dev/docs/queries-search/diagnostics.md)):
```bash
omnigraph lint --query queries.gq --schema schema.pg --json
```
`Q000` identifies a file the parser refused, at `line , column `; a
[branch statement](https://www.omnigraph.dev/docs/branching.md) where declarations were expected
and a [settings line](#session-settings) naming an unknown setting or a value
outside its row report the same way. `L201` warns when a nullable property is
never set by any update query in the inspected set. Type errors report the
affected query and their `T…` code. The command exits nonzero when the overall
status is an error.
For every query that compiles successfully, JSON output includes an
`operation` descriptor:
* `result` lists projected fields in return order. Each field has `name`,
`kind`, and `nullable`; list fields also have `item_kind`, and vectors
have `vector_dim`.
* `reads` conservatively lists every node or edge type the query may inspect.
* `writes` lists every node or edge type the query may change and is empty
for a read query.
Read and write entries are sorted, deduplicated objects with `kind`
(`node` or `edge`) and the case-sensitive `type_name`. Result kinds use
the spellings `string`, `bool`, `int`, `bigint`, `float`, `date`,
`datetime`, `blob`, `vector`, `list`, and `object`. A parse or type
error has no `operation` descriptor.
A mutation target appears in both `reads` and `writes`. An edge insert also
reads its endpoint node types, and a node delete includes incident edge types
that its cascade may remove.
# Search
Source: https://www.omnigraph.dev/docs/queries-search/search
OmniGraph combines vector, full-text, and graph patterns in one `.gq` query.
Search expressions can filter or order a matched node set; a `limit` is required
for nearest-neighbor ordering.
## Functions
| Function | Meaning |
| ---------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
| `nearest($d.embedding, $q)` | Rank vectors by L2 distance. `$q` may be a vector or text that the configured embedding provider converts to a vector. |
| `search($d.body, $q)` | Full-text token search. |
| `fuzzy($d.body, $q [, max_edits])` | Full-text search with edit-distance tolerance. |
| `match_text($d.body, $q)` | Match a full-text query in a `match` block. |
| `bm25($d.body, $q)` | BM25 relevance score. |
| `rrf(rank_a, rank_b [, k])` | Fuse two rankings with Reciprocal Rank Fusion. The default `k` is 60. |
Search arguments use their declared types. `fuzzy` defaults `max_edits` to 2
when omitted; a supplied value must be an integer from 0 to 4,294,967,295.
`rrf` defaults `k` to 60 when omitted; a supplied value must be an integer
from 1 to 4,294,967,295. A supplied null is an error, including a nullable
parameter that was omitted from the parameter map. Query text and vectors
must also be non-null; vector elements must convert to finite F32 values.
Filters in the `match` block are applied before ranking, so `limit 10` means the
top ten matches that satisfy the graph and property filters.
A `bm25()` ordering reads every matching entity before applying the final
limit. Equal scores are then ordered by entity identity, including when the
smallest identity lies beyond the first search candidates.
Full-text rankings inside `rrf()` are
never bounded this way: each full-text arm scans every matching entity, and
fusion ranks the entities that satisfy the graph and property filters, so
bounding an arm could silently drop an entity's contribution and shift fused
results. When a traversal constrains the ranked variable and the graph shows
few entities could satisfy it, the full-text arms instead rank only those
entities (an unbounded, index-served prefilter — results are identical, the
scan is just smaller); broad traversals keep the full scan. A `nearest()`
ranking inside `rrf()` is inherently top-k, as vector
search always is: an entity outside its window adds no vector contribution
to its fused score, so a traversal that drops the window's top matches can
shift fused ranks.
## Vector search
```graphql
query similar($q: Vector(4)) {
match { $d: Document }
return { $d.slug, $d.title }
order { nearest($d.embedding, $q) }
limit 10
}
```
Raw vectors are ranked with L2 distance. Vectors produced by OmniGraph's
embedding client are normalized, so L2 and cosine similarity produce the same
ordering for those generated vectors. See [Embeddings](https://www.omnigraph.dev/docs/queries-search/embeddings.md) for text
queries and provider configuration.
A `nearest` scan uses a vector index only when the index ranks exactly by L2:
the IVF\_FLAT index with the L2 metric that Omnigraph builds. A vector index
written another way, for example by another tool (another metric, quantized
vectors or HNSW), is not used: the scan ranks every row flat, which is exact
and reads the whole column.
IVF vector searches keep Lance's adaptive one-partition minimum and cap the
partitions a `nearest` scan reads at 20 per index delta by default. The cap
prevents Lance's centroid-distance heuristic from expanding a small search
across the entire index; larger values can improve recall at the cost of
latency and object-store I/O.
A capped scan that returns fewer candidates than requested with partitions
left unread is rerun with the cap raised four-fold, then without a cap, until
the candidates are found or the index is exhausted. A scan that ended short
for any other reason (every matching row found, the whole type read) is not
rerun. When a traversal or a filter above the scan drops candidates and
leaves `limit` unfilled, the query asks the scan for four times, then
sixteen times, the requested candidates; a query whose survivors are rarer
than one in sixteen of the nearest candidates then runs one exact pass over
the whole type (every row with an embedding ranked, no cap), taken sooner
once a rung would cover the whole type anyway, so `limit` is
filled whenever that many survivors exist; a query whose survivors are permanently fewer than
`limit` pays that whole-type pass on every execution. A `nearest` ordering constrained by a traversal is first restricted
to the entities that can satisfy the traversal's first hop when few entities
can (the same gate `rrf()` uses), and ranks only those. A `nearest()` arm
inside `rrf()` is a top-k window: the arm's scan widens its own cap, but a
traversal that drops the arm's rows shortens the fused answer. As with every
IVF search, a full candidate count does not make the ANN ranking exact; the
cap remains a recall/latency tradeoff.
| Setting or variable | Meaning |
| ------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `ann_nprobes` ([session setting](https://www.omnigraph.dev/docs/queries-search/queries.md#session-settings), `request` scope; process default `OMNIGRAPH_ANN_NPROBES`) | Partition cap per index delta of a `nearest` scan; default 20, `0` removes the cap, an invalid value refuses startup |
| `OMNIGRAPH_RRF_GATE_RATIO` | Fraction of the ranked type below which a traversal-constrained `nearest` or `rrf()` prefilters its scan; default 0.10, `0` turns the gate off, an invalid value is the default |
| `OMNIGRAPH_RRF_GATE_MAX_IDS` | Largest eligible set the gate pushes into the scan; default 100000, `0` turns the gate off, an invalid value is the default |
| `rrf_plan` ([session setting](https://www.omnigraph.dev/docs/queries-search/queries.md#session-settings), `process` scope; process default `OMNIGRAPH_RRF_PLAN`) | `auto` (default), `force_prefilter`, or `force_postfilter`, for diagnosis. On a traversal-constrained `nearest`, `force_postfilter` can leave `limit` unfilled and `force_prefilter` ranks the eligible entities regardless of the size threshold |
## Full-text search
Use full-text functions for token search, fuzzy terms, and relevance. Use the
query language's exact `contains` and `starts_with` predicates for literal,
case-sensitive substring and prefix matching.
A full-text function needs a full-text index on its property: a one-column
`@index` or `@key` on a free-text String property of a node. The index's
analyzer folds case and stems, so `"deep"` matches `"Deep Learning"`. A call
on any other String property is refused at type checking (`T27`); for an
enum, compare its values with `=` instead. The check reads only the schema.
`search`, `match_text`, and `fuzzy` are positive predicates in `match`: use
a standalone call or compare the call to literal `true` with `=`. Other
comparisons, including `= false`, `!= true`, a Boolean parameter, and a
call on the right side, are refused at type checking (`T38`).
Separate `not` blocks may reuse a local variable name for different node
types. Each full-text call uses the index on the type and property bound
within its own block.
```graphql
query relevant($q: String) {
match { $d: Document }
return { $d.slug, bm25($d.body, $q) as score }
order { bm25($d.body, $q) desc }
limit 10
}
```
`score` is the relevance the ordering used, one computation observed twice:
the projected `bm25(...)` must repeat the leading `order` key (`T33`), and
`nearest(...)` projects its distance the same way; that distance is the
squared L2 distance Lance ranks by. Without an alias the column is
`d._score` or `d._distance`. `rrf(...)` projects the fused score under the
same rule, an `F64` named `d._rrf` after its first arm's binding without an
alias. A `nearest(...)` or `bm25(...)` that appears only as an `rrf` arm is
refused (`T33`), and so are the predicates `search(...)`, `fuzzy(...)` and
`match_text(...)` (`T35`). A relevance ranking orders most relevant first:
`asc` on a `bm25(...)` or `rrf(...)` order key is refused (`T54`), as any
modifier on `nearest(...)` is at parse time.
Exact String predicates remain correct without an index. A free-text index does
not accelerate equality, `starts_with`, or literal substring `contains`.
## Hybrid ranking
Reciprocal Rank Fusion combines rankings without assuming their raw scores use
the same scale:
```graphql
query hybrid($vector: Vector(4), $text: String) {
match { $d: Document }
return { $d.slug, $d.title }
order { rrf(nearest($d.embedding, $vector), bm25($d.body, $text)) }
limit 10
}
```
Ranking order is a contract, not a side effect: search-ordered results are
sorted on the search score itself, including through multi-hop traversals, with
secondary keys and the entity-id tie-break applied after the score. A fused
order sorts every row by the fused score, then the keys written after
`rrf(...)`, then every binding's id, and `limit` counts rows after that order;
an entity that fans out to several rows contributes each of them. The full
ordering contract lives on the [queries page](https://www.omnigraph.dev/docs/queries-search/queries.md).
## Ranking a traversal-reached binding
A search order may rank any binding of the pattern, whichever binding the
`match` block declares first, and whether or not the ranked binding is
declared at all:
```graphql
query passages($matter: String, $q: String) {
match {
$r: SourceRevision { matter_number: $matter }
$r hasPassage $p
}
return { $r.path, $p.locator }
order { bm25($p.text, $q) }
limit 5
}
```
The ranking starts the traversal: the query scans every `Passage`, ranks it
and keeps the passages the traversal reaches from the selected revision, in
score order. Ranking a destination therefore costs a scan of the ranked type,
however few entities the other end reaches. A `nearest()` ranking ranks the
reached entities, never a global top list that the traversal later cuts.
An `rrf()` whose two arms rank two different bindings one traversal connects,
directly or over several hops, is refused at type checking (`T59`): a ranking
starts the traversal at the binding it ranks, and one traversal cannot start
at both. Rank one binding in both arms, or bindings no traversal joins.
## What a ranked read reports
A read whose order is a search carries a `ranking` object in its JSON
envelope (`POST /query`, a stored-query call, `omnigraph query --json`, and
the metadata line of `--format jsonl`). For `hybrid` above, with
`rrf(...) as score` in `return`:
```json
"ranking": {
"by": "rrf",
"order": "descending",
"exhaustive": true,
"score_columns": ["score"]
}
```
`by` is the search function the order names: `nearest`, `bm25` or `rrf`.
`order` says which way the score ranks: `ascending` for a `nearest` distance,
where smaller is nearer, and `descending` for a `bm25` or fused score, where
larger is more relevant.
`exhaustive` says what this run did. When it is `true`, no row the search
admitted and did not return ranks above a returned row. `bm25` scores every
match, so it is always exhaustive. A `nearest` search is exhaustive when it
searched every partition of the vector index, scored every row without one,
or returned every row it admitted. It is `false` when the `ann_nprobes` cap
left partitions unsearched, so a nearer row may exist; raising the cap
searches more. An `rrf(...)` is exhaustive when its `nearest` arm is.
`score_columns` lists the returned columns that hold the ranking's own
score, in return order. Two columns that return one score both appear; a
column computed from the score, such as `bm25(...) > 0.0 as hit`, does not.
It is empty when the read returns no score. The object never carries a query
text, vector or other parameter value, and a read whose order is no search
has no `ranking`.
## Indexes
`@index` and `@key` declare index intent. For a single-property node declaration,
OmniGraph currently creates:
| Property | Index use |
| ---------------------------------------- | --------------------------------------------- |
| Enum, number, Boolean, Date, or DateTime | Equality, range, membership, and null filters |
| Free-text String | Full-text functions |
| Vector | `nearest` |
Node ids and edge ids/endpoints are indexed automatically. Lists and Blobs do
not receive property indexes. Composite declarations and edge-property
declarations do not currently create property indexes.
Indexes are derived performance data. A new declaration may still be pending,
and newly written entities may fall outside existing coverage. Queries remain
correct by scanning missing or uncovered data: full-text functions match the
scanned rows with the index's analyzer, and vector search falls back to an
exact scan when needed. Every write records a declared full-text index's
analyzer before its postings are built. A table version written by an older
release may hold rows and no part of the index at all; a full-text call on
it is refused before any scan with HTTP `409` and a
`full_text_index_required` detail naming `Type.property`, never answered
with a case-sensitive scan. Build the index to lift it:
`omnigraph index build --target --branch `, on any branch.
Historical snapshots of that version stay refused. Run:
```bash
omnigraph optimize --target ./graph.omni
```
after a large load or merge, and on a regular maintenance cadence, to refresh
coverage and compact data on `main`. On another branch, `index build` builds the
branch's missing indexes and refreshes lagging BTREE and vector coverage; see
[Build indexes](https://www.omnigraph.dev/docs/operations/maintenance.md#build-indexes). An empty vector property remains pending until it has a
non-null vector to index.
# Session settings
Source: https://www.omnigraph.dev/docs/queries-search/settings
A file may open with settings lines, before its declarations or its one
statement. `set = ;` gives one setting a value for that file:
the CLI applies it to the invocation, the server to the request, and nothing
is persisted, so the next file starts from the process defaults again. The
same name set twice in one file takes the last value. `reset ;`
restores one setting's process default; `reset all;` restores every setting
the caller may set. `show ;` and `show all;` are themselves the file's
one statement, run through `omnigraph query` or `POST /query`, and return one
row per setting with the columns `name`, `value`, `default`, `source` and
`scope`. `source` is `default`, `env` (a process default read from the
environment), `request` (the `settings` field, a `set` query parameter or
`--set`) or `file` (a `set` line). A file of only settings lines, `set` or
`reset`, `reset all;` alone included, is refused:
`a file of only settings lines carries no statement`.
```graphql
set merge_lineage = verify;
query reports($who: String) {
match { $p: Person { name: $who } $p reportsTo{1,3} $m }
return { $m.name }
}
```
```graphql
reset merge_lineage;
branch merge review into main;
```
```graphql
show all;
```
`show` is authorized as a scope-free read, the decision `branch list` takes: a
credential scoped to one branch, or one holding only change permissions, is
refused `show`. A graph-wide reader sees every row, process rows included, and
each row's `source` says whether the value came from the environment.
Every setting has a scope. A `request` setting is set by any caller. A
`process` setting is set only by the process that hosts the engine: the
server from its environment, and a direct CLI run (`--target`) from its
environment, its `--set` values and the file; a served caller's `settings`
field, `set` query parameter or `set` line naming one is refused.
On a server, the resource budgets `traversal_work_limit`, `query_memory_bytes`,
`write_max_bytes` and `load_source_max_bytes` also have a ceiling: the larger of
the server's process default and the built-in default. A request may set any
value up to it, lowering a budget or restoring one an operator lowered; a value
above it is refused before anything runs, and the operator raises the ceiling
through the setting's environment variable. Embedded sessions and direct
`--target` CLI runs have no ceiling.
A setting reaches the engine through one of three doors:
* CLI: `--set name=value`, repeatable, on `omnigraph query`, `mutate`,
`branch merge`, `commit changes` and `changes poll`, and direct `--target` loads; see the
[CLI guide](https://www.omnigraph.dev/docs/cli.md#session-settings).
* HTTP body: the `settings` object on `POST /query`, `POST /mutate`,
`POST /mutate/if-graph-commit` and `POST /branches/merge`, one key per
`request` setting (`{"settings": {"merge_lineage": "verify"}}`); an unknown key
is refused.
* HTTP query string: `set==`, repeatable, on `GET /changes` and
`GET /commits/{id}/changes`.
A `set` line in the text applies after the door's value, so the file wins.
A stored query runs under the process defaults: `omnigraph query `
without `-e` or `--query` refuses `--set`, and `POST /queries/{query_name}` takes no
`settings` field.
Each setting's environment variable is its process default: the server reads
it once at startup, the CLI once per run, and the value is what every request
starts from and what `reset` returns to. An invalid or out-of-range value
refuses startup; no default is substituted.
| Name | Type and values | Default | Scope | Process default variable | What it chooses |
| ------------------------- | -------------------------------------------------- | ----------------------------------------- | ------- | --------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `engine` | enum `v2` | `v2` | request | `OMNIGRAPH_ENGINE` | the engine a read query runs on; `v2`, the plan runner, is the only value, and `v1` is refused as an unknown value; this setting does not change change-feed or merge execution |
| `rrf_plan` | enum `auto`, `force_prefilter`, `force_postfilter` | `auto` | process | `OMNIGRAPH_RRF_PLAN` | the reciprocal rank fusion plan on a traversal-constrained `nearest`, for diagnosis |
| `merge_lineage` | enum `off`, `on`, `verify` | `on` (a debug build defaults to `verify`) | request | `OMNIGRAPH_MERGE_LINEAGE` | how a merge finds the entities it classifies: the full-scan walk, the lineage path, or both compared |
| `change_lineage` | enum `off`, `on`, `verify` | `on` (a debug build defaults to `verify`) | request | `OMNIGRAPH_CHANGE_LINEAGE` | how a graph diff or commit page finds the entities it compares across a window with no adjacent transaction: the exact ordered merge of both versions, the entities paired by stable identity, or both compared |
| `ann_nprobes` | integer, at least `0` | `20` | request | `OMNIGRAPH_ANN_NPROBES` | the partition cap per index delta of a `nearest` scan; `0` is no cap |
| `stage_write_concurrency` | integer `1..=64` | `8` | process | `OMNIGRAPH_LOAD_CONCURRENCY` | the width of the staged-write fan-out for `load` and `mutate`: tables staged at once, and in one `load csv` overwrite the Lance writers and the streams that parse the source (one per 8 MiB of it, at most this many) |
| `traversal_work_limit` | integer `1..=9223372036854775807` | `1000000` | request | `OMNIGRAPH_TRAVERSAL_WORK_LIMIT` | shared traversal row-work cap for statements containing alternatives or wildcard |
| `write_max_bytes` | integer, at least `1` | `33554432` (32 MiB) | request | `OMNIGRAPH_WRITE_MAX_BYTES` | independent row-data and Blob-payload allowances for bounded writes; changes the existing mutate, load and materializing branch-merge limits; a value raised above the default is held in memory as one typed batch at an overwrite load's commit; a `load csv` statement applies it to each staged-write input: an overwrite stages the reader's 8,192-row batches and refuses a batch over it as `staged_write_input_bytes`, while append and merge keep the keyed adapter's ceilings |
| `load_source_max_bytes` | integer, at least `1` | `33554432` (32 MiB) | request | `OMNIGRAPH_LOAD_SOURCE_MAX_BYTES` | the largest object a `load csv` statement reads, checked before any payload byte is fetched; a served load is additionally bounded by the server's workload admission |
| `query_memory_bytes` | integer, at least `1` | `157286400` (150 MiB) | request | `OMNIGRAPH_QUERY_MEMORY_BYTES` | the memory pool one read query runs under; operators that cannot spill refuse work beyond it, while spillable operators may use bounded scratch storage; raising it permits more tracked memory per query; a load statement additionally admits its read-reuse buffers (a statement-scoped session, held table handles and a byte-range memo) against a bounded share of this allowance, and runs its reads uncached when the share cannot be admitted; it is also the allowance of one `load csv` statement's tracked memory; an overwrite admits it once, before any partition reads: what the statement already holds (the probed source, the swept key sets) comes off the top and the rest is shared out, 52 of 64 parts equally across the parse partitions (60 when the type has no unique check), 8 to the unique holders, 4 uncharged for lease bookkeeping and the fragment sink's open chunks, a copy of rows each partition's reader already charged; a partition's share holds its source pieces (8 MiB each, admitted before the parse), its decoding and every batch it decoded twice over (the batch and the sink's copy of it) with that batch's validation facts, all charged by the partition's own reader in source order, so whether the load refuses and the `actual` it reports (the smallest limit that would have admitted the charge) are the same on every run; a share under the partition's source admission plus one slice refuses before anything is read, and a refused run of several partitions is rerun as one partition holding the whole partition share before its refusal is reported; a partition needs roughly its source range plus twice its typed rows within its share; an append or merge captures the source once per statement and reuses it on every reprepare attempt; JSONL loads keep their existing bounds |
| `history_release_bytes` | integer `1024..=262144` | `262144` | request | `OMNIGRAPH_HISTORY_RELEASE_BYTES` | the byte budget of a branch's buffer of unreleased commits; a mutate, load or branch merge of the session whose buffer and head reach it closes a history block, and the publish after it writes the block under `__history` (schema apply, branch create and delete and upgrade publish under the production budget); a load reaches it only through the CLI or an embedded `Session`, since the HTTP load routes take no settings; query results never change, only the number of requests and files; production keeps the default, a caller may only lower it |
| `explain_dump_dir` | text | empty (disabled) | process | `OMNIGRAPH_EXPLAIN_DUMP_DIR` | the directory each finished explain document is written into, one `_.explain.json` per operation and logical hash; each complete document atomically replaces the previous one at that path; empty disables the write; a failed write reports on stderr and never fails the operation |
For these statements the cap covers all traversals, including named and nested
ones, and remains shared across retries. It charges consumed source rows,
conservative physical edge rows before each scan, examined adjacency entries
and bound-edge source replication. Exceeding the cap fails with
`traversal_work_limit`; a result limit cannot bypass admission. This is a row-work
bound, not a byte, elapsed-time or total-query CPU/I/O bound. Memory and scratch
retain their existing limits. The budgeted route uses pinned Lance scans with
indexed filtering where available.
Sources are admitted in fixed windows of at most 8,192 rows, independent of
upstream batch boundaries. Each nonempty frontier probe charges the selected
table's full physical row count before opening the scan. Multiple windows,
members, directions and hops can therefore charge a table repeatedly. A directed
one-hop query over selected tables totaling 1,000,000 physical rows exceeds the
default cap even when its start node has only one neighbor. Index selectivity
does not reduce this conservative admission charge. The cap does not bound
index decoding, bytes read, or storage latency.
Internal traversal pins used by the GQT harness can force indexed execution. A
forced CSR pin is refused for statements with selections; there is no public
`set traversal` setting.
`write_max_bytes` applies to each existing write account: row data and Blob
descriptors, logical Blob payloads, parser estimates and retained removed IDs.
The accounts keep their existing per-table, per-operation or per-chunk scope.
Row data and Blob payloads have separate allowances: a payload exactly at the
limit can accompany a row whose metadata fits its own allowance. The value is
captured once for an operation, including its retries and merge publication.
A one-row keyed write costs about 1 to 3 KiB of row data from Arrow buffer
rounding, so allowances below a few KiB refuse every write.
Strict inserts and overwrite writes also pass `write_max_bytes` to Lance as
the buffer accumulation target for each data-file writer. Lance divides it
across top-level columns instead of using its default 8 MiB per column.
The default 32 MiB target therefore raises or lowers each column's threshold
depending on the number of columns.
This is a flush threshold, not a total memory cap: the incoming array can
cross it, nested fields and encoding have additional working memory, and
concurrent table writers each receive the target. Upserts and updates use
Lance merge-insert, which does not expose this option and keeps its native
buffering defaults. Row and Blob admission limits still apply to those writes.
Neither byte setting has a fixed maximum. A lower environment value is a
process default, so a request may still select any value of at least `1` up to
the server's ceiling (see above).
Raising `write_max_bytes` above the default admits a larger staged-write
input (a JSONL load's typed batch, a CSV overwrite's reader batch), held in
memory at commit; raising `load_source_max_bytes` admits a
larger `load csv` source object into memory before parsing. On a server both
stay bounded by workload admission (`OMNIGRAPH_GLOBAL_BYTES_MAX`,
`OMNIGRAPH_PER_ACTOR_BYTES_MAX`). Embedded sessions and direct `--target` CLI
loads can override them; a `load csv` statement travels on `/mutate`, whose
`settings` field and `set` prefix apply. The JSONL HTTP load routes inherit
the process default and accept no request setting; remote CLI loads of those
routes refuse `--set`. HTTP bodies, Blob reads, compaction and external URI
metadata retain their independent limits. Lower values can produce smaller
merge chunks or reject a write that the default admits.
`history_release_bytes` is measured against the branch's buffered commit and
table-change rows as stored, plus the head whole; the row layout is in
[storage versioning](https://github.com/ModernRelay/omnigraph/blob/main/docs/dev/versioning.md#current-storage-contract). The
commit that reaches the budget records the decision in its commit id's slot,
so sessions publishing to one branch under different budgets agree on where a
block closes, and the publish after it releases the block whatever its own
budget. A load carries the setting only from the CLI or an embedded
`Session`: the HTTP load routes take no settings. The bound on a head's
commit fields stays 256 KiB, the production budget. A lower budget means more
and smaller `__history` files and more requests per release, never different
rows.
A name outside the table, a value of the wrong type, and a value outside the
declared values or range are each refused with the table's row. A `process`
setting named in a request is refused too: the `settings` field has no member
for one, so the key is refused as unknown, while a `set` line in the text and
a `set=` parameter answer the row's message. `omnigraph lint` reports a
settings line's refusal as `ERROR line , column : `.
```text
set merge_lineage = fast;
error: unknown value `fast` for setting `merge_lineage`; expected one of off, on, verify
set traversal = csr; (likewise reset traversal; and show traversal;)
error: unknown setting `traversal`; expected one of engine, rrf_plan, merge_lineage, change_lineage, ann_nprobes, stage_write_concurrency, traversal_work_limit, write_max_bytes, load_source_max_bytes, query_memory_bytes, history_release_bytes, explain_dump_dir
set ann_nprobes = "many";
error: setting `ann_nprobes` takes an integer of at least 0, got a string
set stage_write_concurrency = 0;
error: setting `stage_write_concurrency` takes an integer in 1..=64, got 0
set stage_write_concurrency = 64; (in an HTTP request)
error: setting `stage_write_concurrency` is a process setting; it is read from the server's environment, not from a request
```
See [Query language](https://www.omnigraph.dev/docs/queries-search/queries.md) for query syntax.
# Traversal and match patterns
Source: https://www.omnigraph.dev/docs/queries-search/traversal
Inside `match { ... }`:
| Pattern | Meaning |
| ------------------------------------------- | --------------------------------------------------------------------------------------------- |
| `$p: Person { name: $name }` | Bind nodes and filter properties. |
| `$person worksAt $company` | Follow a directed edge. |
| `$a knows{1,3} $b` | Follow a path from one to three hops. |
| `$a $b` | Match the edge in either direction. The edge must connect the same node type at both ends. |
| `$a $rel:related $b` | Bind a single-hop edge instance so its properties can be used. |
| `$a (knows \| likes) $b` | Follow one hop across either concrete edge type. |
| `$a *{1,3} $b` | Follow one through three hops across compatible edge types; declare both endpoint node types. |
| `$a $b` or `$a <*> $b` | Follow the selected edges in either direction. |
| `$a $e:(knows \| likes) $b` or `$a $e:* $b` | Bind each concrete selected edge. |
| `$p.age >= 18` | Apply a filter expression. |
| `not { $p blocked $other }` | Keep rows for which the inner pattern has no match. |
| `exists { $p blocked $other }` | Keep rows for which the inner pattern has at least one match. |
| `count { $a authored $p } > 2` | Keep rows whose inner pattern matches more than two times. |
| `sum($d.size) { $p owns $d } > 100` | Aggregate a property over the inner matches, then compare. |
Omitted bounds always mean exactly `{1,1}`. Explicit `{1,1}` is equivalent.
Multiple hops require a finite `{min,max}` with `1 <= min <= max`; `*` selects
edge types and never means repetition. Duplicate alternatives select a type once.
All members must connect the same source and destination node types, with
direction resolved separately for each member. Wildcards require explicit node
declarations for both endpoint variables in the same lexical block or a visible
outer scope. A declaration in the same block may follow the traversal. Later
outer declarations are not visible inside an earlier nested block. Undirected
traversal and any hop bound above one, on a named edge or a selection, require
the same node type at both ends: a hop ends on the destination type and the
next one must start on the source type, so `$p worksAt{1,2} $c` over
`WorksAt: Person -> Company` is refused with `T5` rather than run as one hop.
Every connected traversal pattern needs an executable endpoint binding; an
unrelated node declaration does not anchor it. Mixed-orientation alternatives
need a declared endpoint type when their orientation would otherwise be ambiguous.
Hop counts are shortest-path distances from the start node: `{2,2}` returns the
nodes exactly two hops away. A node is never re-reached through its own
self-loop or through a cycle back to it. The start node is returned only
through its own self-loop, which counts as one hop, never through a cycle.
An unbound traversal has set semantics for endpoint pairs. Binding the edge
returns one result per matching edge, so parallel edges remain distinct. Edge
bindings are available only for a single hop. Alternatives share the same
deduplication and shortest-distance search across all members. Bound edges are
identified by canonical type plus ID, so equal IDs in different types remain
distinct. An undirected self-loop appears once.
A bound edge property must exist on every member with the same base value type.
Scalar/list shape and Vector dimensions must agree. It is nullable if any
member allows null. Enum domains are combined, and an enum
combined with plain String becomes String. Other conflicting types or missing
properties are static errors. Blob values retain their dedicated read API;
selecting a Blob-bearing edge does not make Blob properties projectable. An empty wildcard returns no rows with the declared node and
edge-metadata columns; ordinary edge properties cannot be inferred.
Wildcard membership is fixed by the invocation's captured catalog and graph
snapshot. Compatible schema additions join later wildcard invocations. An
addition without a referenced common property invalidates that query. Stored-query
validation refuses a schema change that would invalidate a registered query;
with embedded ad-hoc queries, the schema change can succeed and the next query
fails type checking. Existing graph, branch and stored-query permissions apply.
Explicit historical targets refuse wildcard traversal. When traversal opens a
selected edge dataset, absence at that snapshot is an error for named edges and
explicit alternatives alike. Empty-input paths need not open a dataset.
Statements using alternatives or wildcard share a finite
[`traversal_work_limit`](https://www.omnigraph.dev/docs/queries-search/settings.md) across all traversals, members and hops.
Exhaustion terminates the query with an error, possibly after rows were delivered.
Those rows do not constitute a successful complete result. Result `limit` is
optional and does not replace this work budget. Admission includes every selected
table's physical row count before each scan, even for a selective neighborhood;
see the setting for the cost and batching rules.
Each `$_` is a distinct anonymous node: two anonymous traversals from one
variable are independent, so a source with two neighbours matches two by two
rows. Binding a variable a second time (`$p: Person` after `$p` is already
bound, at the top level or inside `not { }`) adds the second binding's
property matches as constraints on the same rows; it never introduces a
second `$p`. Variable names beginning with `__` are reserved.
Bare traversal names begin with a lowercase letter (`worksAt` for the declared
edge `WorksAt`); edge lookup itself is case-insensitive.
Comparison operators are `=`, `!=`, `<`, `<=`, `>`, and `>=`.
See [Query language](https://www.omnigraph.dev/docs/queries-search/queries.md) for expressions, projection and ordering.
# Blobs
Source: https://www.omnigraph.dev/docs/schema/blobs
Use `Blob` properties for bytes that should not be projected through `.gq`.
Blob values can contain graph-managed bytes, an external URI reference, or null.
A valid empty managed Blob is distinct from null.
```graphql
node Document {
slug: String @key
content: Blob?
}
```
## Writing Blob values
Load and mutation input use one String representation:
* `base64:` supplies managed bytes owned by the graph;
* any other String requests an external URI reference;
* `null` stores a null value when the property is nullable.
```jsonl
{"type":"Document","data":{"slug":"manual","content":"base64:SGVsbG8="}}
```
New external references are denied by default. A cluster-served graph must list
allowed URI bases in its graph configuration. Direct `--target` CLI access has no
external-source allowlist, so it accepts managed `base64:` input but rejects new
external references. Credentials must not appear in stored URIs. An allowed
base must lie outside the cluster's storage root and every graph root; see
[External Blob references](https://www.omnigraph.dev/docs/clusters/config.md#external-blob-references). The
same bases admit the source of a `load csv` statement
([CSV loading](https://www.omnigraph.dev/docs/queries-search/mutations.md#csv-loading)), which reads the object once
and stores no reference to it.
Write mode determines ownership:
* `load --mode overwrite` preserves an allowed external URI as an external
reference;
* incremental inserts, upserts, updates, append/merge loads, and branch merges
that write entities copy allowed source bytes into graph-managed storage;
* an existing external reference remains readable even when new external
ingress is disabled, and exportable when it names its whole object (see
below).
An `update` never reads the old value of a Blob it assigns. When it assigns no
Blob and is the only statement of its mutation that writes the type, it can
write just the assigned properties into the rows' existing storage: the row's
Blob and vector values are neither read nor rewritten, and a stored external
reference stays as it is. It does so whenever a matched row holds a stored
external reference, and otherwise when it writes fewer bytes than rewriting
the matched rows whole. That is the usual case when the rows hold large Blobs
or vectors. A one-row update in a large table of small rows rewrites the row.
Such an update is never refused for the Blobs it does not assign: when
rewriting the rows would exceed `write_max_bytes`, or when the mutation also
writes another type, it writes just the assigned properties. Whether it is
admitted never depends on an index, and only the matched rows count toward
`write_max_bytes`, not the other rows stored beside them.
An update that rewrites whole rows carries every Blob cell of a matched row it
does not assign: it reads the cell and rewrites it as managed bytes, and those
bytes count toward `write_max_bytes`. Carrying a stored external reference
therefore needs the graph's external Blob policy to admit the reference's
source. Otherwise the update fails with a 400 that names the type, id, and
property. Assign that property in the same update, to a new value or to null,
to replace or clear the reference without reading it. An update rewrites whole
rows when it assigns a Blob, or when another statement of the same mutation
writes the type.
Only a graph written outside OmniGraph can hold a stored reference to a byte
range of an object. Export writes an external reference as a bare URI, which
reloads as the whole object, so export refuses a ranged reference instead of
widening it. Change-feed images, the change-feed baseline and entity reads by
id describe it exactly, as `{"uri": …, "offset": …, "length": …}` with a
positive `length`, without reading the object. The feed passes the commit that
holds it like any other, and a baseline taken while the row exists succeeds;
that baseline does not reload with `load`. A stored descriptor with an offset
but no length is refused as a Blob integrity error.
OmniGraph never deletes the object named by an external reference.
### Replacing one Blob value
One Blob cell of an existing node or edge, addressed by exact id like a Blob
read, can be replaced or cleared without a load or a mutation. Over HTTP,
`PUT` stores the raw request body as a managed value and `DELETE` clears the
cell:
```http
PUT /graphs/knowledge/blob?entity=node&type=Document&id=manual&property=content&branch=main
Omnigraph-Http-Api: 0.14
Content-Type: application/octet-stream
If-Match: ""
```
An embedded `Session` does the same with `put_blob_at_as` and
`clear_blob_at_as`. A put returns the stored length, its ETag and the commit.
Clearing a cell that is already null publishes nothing and returns no commit.
Each write that changes the cell is one graph commit and needs the `change`
action on the branch. Neither verb inserts a row: a missing entity is
`NotFound` (404). Clearing a property that is not nullable is refused (400).
The write never reads the cell's old value. Lance replaces whole rows, so the row's other cells
are carried as an `update` carries them: a stored external reference among
them needs the external Blob policy to admit its source, and their managed
payloads count toward the operation's Blob payload allowance, `write_max_bytes`,
together with the new value.
A precondition makes the write conditional on the cell's current value: over
HTTP an `If-Match` field, embedded a `BlobPrecondition`. A list of entity tags
(`BlobPrecondition::Tags`) holds when the current ETag is one of them; weak tags
never match. `*` (`BlobPrecondition::AnyExisting`) holds when the cell is not
null. When it fails, the write changes nothing and returns
`BlobWritePreconditionFailed`, over HTTP a `412` `precondition_failed` whose
`blob_precondition_failure.current_etag` and `ETag` header carry the cell's
current ETag when it holds a managed value. A malformed `If-Match` is a `400`. A commit that
lands while the write is in flight makes it start again from the new head and
check again, so a stale ETag fails instead of overwriting the newer value; a
write without a precondition replaces whatever is current. A schema apply or a
delete and recreate of the branch landing in that window refuses the write
instead, over HTTP a `409` `read_set_conflict`.
The returned ETag is the one a read at the returned commit reports. When a call
fails after its commit became durable, for example because its acknowledgement
was lost, the value is published once; read the cell for its current ETag
before retrying with a precondition.
Over HTTP:
* `PUT` takes a `Content-Type: application/octet-stream` body of at most the
server's Blob put limit (the process `write_max_bytes`, at most 32 MiB),
inclusive; a request cannot change it. Another media type is a `415`. A larger body is a `413`, refused
before any of it is read when `Content-Length` declares it, and a body that
does not arrive before the server's body deadline is a `408`;
* both verbs take `branch` (default `main`) and refuse `snapshot` or any other
unknown parameter with a `400`. A missing branch is a `404`;
* `change` is authorized before the body is read;
* success is a `200` with a receipt: `selector`, `branch`, `kind` (`managed` or
`null`), `size` and `etag` for a managed value, `actor_id`, and `commit`,
which is `null` only for a clear of a cell that was already null. A `PUT` also
returns the `ETag` header;
* once the server admits a write it finishes it even if the client
disconnects, so a lost response means the write may have landed: read the
cell before writing again.
From the CLI, `blob put` stores the bytes of `--file` or stdin and `blob clear`
nulls the cell. Both take `--branch`, `--if-match` and `--json`, and print the
receipt the server returns:
```bash
omnigraph blob put node Document manual content --file manual.pdf \
--if-match '""' --target ./graph.omni
omnigraph blob clear node Document manual content --target prod --graph knowledge
```
The CLI refuses input over 32 MiB and a malformed `--if-match` before it
addresses the graph. The value must also fit `write_max_bytes`: the process's
`OMNIGRAPH_WRITE_MAX_BYTES` for a `--target` write, the server's for a served
one. `--as` names the actor of a `--target` write; a served
write takes its actor from the bearer token, and `--as` there only asserts it. A Blob precondition failure exits 4,
embedded or served, as a graph-commit precondition does. Each write is sent
once: after an unknown outcome, read the cell before writing again.
## Query behavior
Blob properties are not ordinary `.gq` read values. They cannot be projected,
filtered, ordered, or aggregated. Write them through load or mutation
assignment, then read an individual Blob value through the dedicated CLI or
HTTP surface.
To replace or clear one value, use the CLI, HTTP or embedded writes in
[Replacing one Blob value](#replacing-one-blob-value); each is an atomic graph
commit.
## CLI reads
The selector is `ENTITY TYPE ID PROPERTY`, where `ENTITY` is `node` or `edge`.
Blob commands address the graph with `--target NAME|ADDRESS`; they do not take a positional graph URI.
```bash
# Stream managed bytes to a file.
omnigraph blob get node Document manual content \
--out manual.bin --target ./graph.omni
# Inspect the value without reading payload bytes.
omnigraph blob stat node Document manual content \
--json --target ./graph.omni
```
Reads default to `main`. `--branch ` and `--snapshot ` are
mutually exclusive.
`blob get` supports `--offset`, `--length`, and `--out`. With no range flags it
streams the complete value. `--offset N` reads from `N` to the end;
`--length M` reads the first `M` bytes; together they read `N..N+M`. A length of
zero is rejected. An end beyond the value is clamped to EOF, while a start at or
beyond the end of a non-empty value is unsatisfiable.
A failure before transfer begins leaves an existing `--out` file untouched. A
successful empty value creates or truncates the file. If transfer fails after
output begins, stdout or `--out` may contain the prefix already written. For
atomic file replacement, write to a temporary path, check the exit status, then
rename it.
`blob stat --json` reports the selector, value kind, and exact resolved snapshot.
A managed value also reports `size` and `etag`; a whole-object external value
reports its stored `uri`. Fields that do not apply are omitted rather than set
to null.
The CLI never follows an external URI. `blob get` refuses a whole-object
reference and points to `blob stat`, which returns it without opening the
target. A persisted ranged external descriptor is rejected by both commands;
OmniGraph never widens it to the complete target object.
## HTTP reads
Servers expose the same logical selector with GET and HEAD:
```http
GET /graphs/knowledge/blob?entity=node&type=Document&id=manual&property=content&branch=main
Omnigraph-Http-Api: 0.14
```
Both methods follow the [HTTP contract](https://www.omnigraph.dev/docs/operations/server.md#http-contract),
including checking the response header before consuming bytes.
Use `snapshot=` instead of `branch` for an immutable historical read.
For managed values:
* GET returns bytes; HEAD returns the same metadata without a body;
* a single standard `Range` request returns `206`, or `416` when unsatisfiable;
* `ETag`, `If-Match`, and `If-None-Match` support conditional delivery;
* the response identifies the exact resolved graph snapshot.
Treat ETags as opaque validators for the selected graph representation, not as
content hashes. An unrelated change to the same entity type can produce a new
ETag even when this cell's bytes are unchanged.
For a whole-object external value, GET and HEAD return `302` with the stored URI
in `Location`. The server does not fetch, sign, authorize, or proxy that object
and does not claim its size or ETag. A persisted ranged external descriptor
fails loudly instead of redirecting to a wider value.
## Limits
Blob limits bound the memory one operation needs. An operation over a limit
fails before it changes the graph. Over HTTP it returns `413` with a
`resource_limit` detail naming the `resource`, its `limit` and the `actual`
value observed. For a write, the CLI reports the same three fields; the
`omnigraph blob` commands report an over-limit range read as
`Blob read range exceeds the limit`. Split the work into smaller operations
and retry.
| Limit | Applies to | Reported resource |
| ----------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `write_max_bytes` decoded bytes | Each node or edge type in one load, in every mode, including `overwrite` | `decoded blob input bytes for
` |
| `write_max_bytes` decoded bytes | One `base64:` value in an insert or update mutation; a statement's values add up across its Blob properties and an update's matched rows. A load reports its decoded bytes under the `for
` and `per operation` names | `decoded blob input bytes per operation` |
| `write_max_bytes` row bytes per type and across all touched types | Incremental writes: `append` and `merge` loads, inserts and updates. Counts ordinary columns, Blob descriptors and Arrow bookkeeping; excludes logical Blob payload buffers | `keyed write bytes for
`, `keyed entity bytes for
`, `retained keyed batch bytes per operation`, `keyed parsed entity bytes for
`, `keyed parsed entity bytes per operation` |
| `write_max_bytes` logical Blob payload bytes | One incremental write across all types: inline payloads, copied external payloads and Blob values carried by updates or by a Blob put count together | `materialized blob payload bytes`, `decoded blob input bytes per operation` |
| `write_max_bytes` bytes, inclusive | The bytes of one Blob put, embedded or the body of an HTTP `PUT`, checked before the write opens a table | `Blob write payload bytes` |
| `write_max_bytes` Blob payload bytes | One branch merge that writes rows, across all types, managed and external bytes together | `materialized blob payload bytes` |
| 8,192 external references | One write operation or merge | `external Blob reference cells` |
| 32 MiB of retained URI metadata | One write operation or merge. Every copy of a URI the operation keeps counts, plus 24 bytes per copy: admission keeps each reference's text twice and each distinct object's normalized URI twice, so distinct URIs reach the limit at about 8 MiB of text | `external Blob URI metadata bytes` |
| 64 KiB | One external URI | `external Blob URI bytes` |
| 4 MiB | One embedded managed range read | `Blob read range bytes` |
Superseded versions of a key written in one operation count as input
(`decoded blob input bytes per operation`) until the last-write-wins fold.
An update of a row inserted earlier in the same mutation is such a version:
it carries the pending row's Blob, so that payload counts twice, the carried
copy as `materialized blob payload bytes`.
[`write_max_bytes`](https://www.omnigraph.dev/docs/queries-search/settings.md) defaults to 32 MiB and accepts any
integer of at least `1`; a Blob put stays bounded at 32 MiB by the put limit
whatever the setting. Payload and row allowances are independent: a payload exactly
at the limit fits when its row and descriptors also fit. Repeated external
references each count their payload length even when one fetch serves them.
External-only admission can also report `materialized external blob payload bytes`.
`
` names the type as `node:` or `edge:`, for example
`keyed entity bytes for node:Document`.
The HTTP load request body is also capped at 32 MiB. That cap counts the
encoded request, so one request carries about 24 MiB of decoded `base64:`
data. A Blob `PUT` carries raw bytes, not `base64:` text, so its 32 MiB body
holds a full 32 MiB value. Every HTTP request other than a load (`/load` and
`/load/ndjson`) or a Blob `PUT` is bounded by the default 1 MiB request body
limit, so a `base64:` literal in an HTTP mutation hits that limit first.
Values larger than these limits stay readable. The CLI and the HTTP server
read managed values in 4 MiB ranges, so a large value streams without a
whole-value buffer, and one HTTP response holds at most two ranges at a time.
`omnigraph optimize` bounds the Blob payload of each compaction batch
separately; see [Optimize](https://www.omnigraph.dev/docs/operations/maintenance.md#optimize).
## Lifecycle
Blob readers stay pinned to the snapshot selected when they were opened. They
never switch to newer bytes when a branch advances. Branch deletion and
destructive cleanup can remove storage needed by a long-running reader; quiesce
those readers first when they must finish reliably.
Historical reads are fail-closed. Type renames remain addressable through the
current type name, but a historical property rename, drop/re-add, or branch
incarnation may be refused when the selected snapshot does not carry enough
identity information to prove it is the same logical Blob property. OmniGraph
returns an error rather than guessing from a reused name or physical field
position.
# Schema Language (`.pg`)
Source: https://www.omnigraph.dev/docs/schema
A schema declares the node and edge types a graph accepts. OmniGraph validates
the same schema and constraints for mutation queries, loads, and branch merges.
Actor attribution on a commit does not create a graph node. Applications may
declare their own `Actor`, `OmniActor`, or other provenance types; these are
ordinary customer-owned types with no automatic rows or permission grants.
```graphql
node Person {
email: String @key
display_name: String
age: I32?
@range(age, 0..150)
}
node Company {
slug: String @key
name: String
}
edge WorksAt: Person -> Company @card(0..) {
role: String?
@unique(@src, @dst)
}
```
Comments use `// ...` or `/* ... */`.
## Declarations
```text
interface { }
node [implements , ...] { }
edge : -> [@card(...)] { }
```
Interfaces provide reusable properties. An edge's endpoint node types must
already be declared. Query traversals spell an edge with a lowercase first
letter (`worksAt` for `WorksAt`); lookup is otherwise case-insensitive.
## Property types
| Type | Values |
| ----------------- | ----------------------------------------------------------------------- |
| `String` | UTF-8 text |
| `Bool` | `true` or `false` |
| `I32`, `I64` | Signed integers |
| `U32`, `U64` | Unsigned integers |
| `F32`, `F64` | Floating-point numbers |
| `Date` | Calendar date |
| `DateTime` | Timestamp with millisecond precision |
| `Vector(N)` | `N` 32-bit floating-point values |
| `Blob` | Managed bytes or an external reference; see [Blobs](https://www.omnigraph.dev/docs/schema/blobs.md) |
| `enum(a, b, ...)` | One of the declared strings |
| `[T]` | A list of scalar `T` values |
| `T?` | A nullable value |
Property names starting with `_` are reserved for system columns and are
rejected when a schema is admitted. The reserved namespace covers Lance's
virtual system columns (`_rowid`, `_rowaddr`, `_rowoffset`,
`_row_created_at_version`, `_row_last_updated_at_version`) and OmniGraph's
own implicit stored columns, spelled `__id` on nodes and edges and
`__src`/`__dst` on edges for newly created graphs.
Supported existing graphs without the `system-columns` feature keep the earlier spellings `id`, `src`,
and `dst` for their implicit columns. They reserve `id` on nodes and edges,
and `src`/`dst` on edges. On current graphs all three names are ordinary
property names. Query meta-fields use `@id`, `@src`, and `@dst`; projected
node objects use `@id`, and JSONL exports use a top-level `id`. The schema
endpoint (`GET /schema`) reports the physical spellings in `system_columns`.
`_distance` and `_score` are also reserved for new declarations: search-ordered
queries rank results by those columns. A graph whose schema already declared
either name before this reservation keeps opening; only new schemas are
refused.
On edge types the names `from` and `to` are also reserved: they are the
insert parameters that name the endpoints.
## Constraints
Constraints can be written in the type body. `@key`, `@unique`, and `@index`
also have a single-property shorthand.
| Constraint | Applies to | Meaning |
| --------------------- | ------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `@key(p, ...)` | node or edge | The property tuple identifies the entity; its id is derived from it. Key properties must be non-null scalar values. An edge key must include both `@src` and `@dst`, and a key may be declared only when the type is created. |
| `@unique(p, ...)` | node or edge | No two entities may share the property tuple. Edge constraints may include `@src` and `@dst`. |
| `@index(p, ...)` | node or edge | Declares index intent. Indexes affect performance, not correctness. |
| `@range(p, min..max)` | node | Restricts a numeric property, or each element of a numeric list, to an inclusive range; either bound may be omitted. Values compare with the bounds exactly at any magnitude, and a decimal bound on an integer property admits the integers on its side: `@range(n, -2.5..2.5)` admits -2 through 2. |
| `@check(p, "regex")` | node | Requires a String property to match the expression. |
| `@card(min..max)` | edge | Bounds how many edges of the type start at each source node; omit `max` for an unbounded range, as in `@card(1..)`. The default is unbounded from zero. See [Edge cardinality](#edge-cardinality). |
`@range(p, ..10)` supplies only a maximum, and `@range(p, 10..)` supplies
only a minimum; both endpoints are inclusive. Integer bounds must fit `I64`
(`-9223372036854775808` through `9223372036854775807`), even for a `U64`
property. Decimal bounds use their `F64` value.
Already accepted graph schemas keep their stored bounds. Older versions
interpreted an upper-only bound as a minimum and converted oversized integer
bounds to `F64`, possibly rounding their value. Opening or upgrading a graph
does not rewrite those accepted meanings. Reapplying that source, or using an
exported schema to create another graph, uses the corrected admission rules:
to preserve the accepted constraint, spell the historical minimum as `min..`
and a stored floating-point bound as a decimal with the recorded value.
Complete outstanding deployments containing affected schema source before
upgrading the binary; deployment source validation uses the admission rules.
`Blob`, list, and vector properties cannot be keys or unique, in the
property-level or the body form. Blob properties also cannot be indexed.
Earlier releases admitted a body-level `@unique` on a list or vector property.
A graph that already declares one still opens, but a write that gives the
property a value is refused, and reapplying that source is refused at
admission. In a cluster, every `cluster apply` of a deployment that contains
such a graph is refused, even one that changes only another graph, query or
policy, until the constraint is removed. To remove the constraint in place, drop every property it names in
one schema change, then add the properties back without it; dropping a
property discards its stored values. A constraint that also names another
property or `@src`/`@dst` cannot be removed in place: rebuild the graph
without it.
Current automatic property indexes are created for single-property node declarations:
orderable scalars and enums receive a scalar index, free-text Strings receive a
full-text index, and vectors receive a vector index. Composite declarations and
edge-property declarations are accepted as schema intent but do not currently
create a property index. Edge endpoints are indexed independently for traversal.
See [Search](https://www.omnigraph.dev/docs/queries-search/search.md#indexes).
### Edge cardinality
`@card(min..max)` counts, for each node of the edge's source type, the edges of
that type that start at it. A write is refused before publication when it
leaves one of these nodes outside the bounds:
* a source node the write inserts or updates, when `min` is above zero;
* a source node whose edges of the type the write adds, removes, or moves,
including the edges an `overwrite` load omits and the edges a node delete
cascades to.
A source node the write deletes is not counted. A node whose type needs edges
(`min` above zero) must therefore arrive in the same write as those edges: one
mutation query that inserts the node and its edges, or one JSONL load that
carries both. A load of the node type alone cannot add such a node, and neither
can a [`load csv`](https://www.omnigraph.dev/docs/queries-search/mutations.md#csv-loading) into it, which carries no
edges.
Schema apply refuses a new edge type with a non-zero minimum whose source node
type already exists (`OG-MF-107`): its existing nodes have none of the new
edges. Declare such an edge with its source type, or without a minimum.
Schema apply widens an existing edge type's range in place: lowering `min`, or
raising or removing `max`, as in `@card(1..1)` to `@card(0..1)`, rewrites no
data, because every stored count inside the old range is also inside the new
one. The new range holds from the next write. Raising `min` or lowering `max`
is refused, because the stored counts would need a check that apply does not
make.
Earlier Omnigraph versions did not count inserted source nodes, so a graph they
wrote can hold nodes below the minimum. Each is counted the next time a write
inserts or updates it or changes its edges of the type, and a load that
rebuilds the graph from an export refuses it. An orphan query lists them:
```graphql
query people_without_employer() {
match {
$p: Person
not { $p worksAt $c }
}
return { $p.email }
}
```
## IDs
Every node and edge has a String id. On the wire it rides at the top level of
the load or export envelope, under the fixed key `id`, beside `type` (nodes) or
`edge`; `data` holds user properties. Legacy-vintage graphs also accept
`data.id` as identity when the top-level `id` is absent; providing both is
refused. In queries the id is the meta-field
`$p.@id` and an edge's endpoints are `$e.@src` and `$e.@dst`
([system fields](https://www.omnigraph.dev/docs/queries-search/queries.md#system-fields)); a bare `id` in a query
is always a user property of that name.
* A node with `@key` derives its id from the complete typed key tuple. Renaming a
key property with `@rename_from` does not change existing ids.
* A node without a key receives a generated id unless input supplies one.
* An edge with `@key` derives its id the same way, in the catalog's key
order: `@src`, then `@dst`, then any scalar members. A composite id encodes
as a JSON array of the member values, for example `["Alice","Bob"]`.
* An edge without a key uses generated or supplied ids. Edges store their
endpoints as the graph's endpoint columns, `__src` and `__dst` on graphs
created by this binary.
For hand-authored load data, omit a keyed node's or keyed edge's top-level `id`
and let OmniGraph derive it; a supplied id on a keyed edge must equal the
derived id exactly. Export includes ids so a graph can be rebuilt without losing edge
references.
## Annotations
* `@rename_from("OldName")` on a node, edge, or property declares a rename
during schema migration. Use the current name everywhere after applying it.
* `@description("...")` adds human-readable metadata to types and properties.
* `@instruction("...")` adds usage guidance to node and edge types.
* `@embed("source_property", model="model-id")` associates a vector with its
source String property. The model is optional; see
[Embeddings](https://www.omnigraph.dev/docs/queries-search/embeddings.md).
Property-level `@key`, `@unique`, and `@index` are shorthands for their
single-property constraints. Unknown annotations are retained as metadata but
have no built-in behavior.
## Schema changes
Always preview a direct schema change before applying it:
```bash
omnigraph schema plan --schema next.pg --target ./graph.omni
omnigraph schema apply --schema next.pg --target ./graph.omni
```
Supported changes include adding types, adding nullable properties, renaming
nodes, edges, or properties with `@rename_from`, adding index declarations,
widening an enum with new values, making a nullable property required, adding
`@unique` to a node type, updating descriptions or instructions, and
dropping node, edge, or property declarations. Dropping a property also drops
every constraint that names only dropped properties: its own `@index`,
`@unique`, `@range` or `@check`, and a composite constraint whose members are
all dropped.
None of these changes rewrites existing rows. Adding a nullable property,
renaming a property, or dropping a property changes only the table's metadata:
existing rows read the new property as null, no stored value or Blob is read,
and the indexes of the remaining properties keep covering the rows they
covered. A dropped property's index is removed by the same commit, and index
maintenance does not build it again. The time and memory
the apply takes therefore do not grow with the number of rows or the size of
their values. They still grow with the table's metadata: its columns, its
indexes and its fragments, of which a table written by many small commits has
more until `omnigraph optimize` compacts them.
Making a nullable property required (on a node or an edge) and adding `@unique`
to a node type read the stored rows first, under the locks the apply holds, and
change nothing if a row breaks the rule: a live row holding a null refuses with
`OG-MF-104`, two rows sharing a key with `OG-MF-108`. The `@unique` check holds
the keys in memory and refuses a table whose keys need more than an estimated
1 GiB. A `@unique` that names a property the same apply adds needs no check,
because every stored row holds a null there and a key with a null part is not
compared. A refused apply leaves the accepted schema in force. A successful one keeps entity
ids, relationships and retained commits, and later writes are checked against
the new rule. To add a required or unique property to a populated type, add it
nullable, load its values, then promote it, with a `@rename_from` in the same
apply if you like. Promotion rewrites no data file, with one exception: a
deleted row keeps its value in storage, so a fragment whose deleted rows hold a
null in the promoted property is rewritten without them. Making a property
required also makes its storage column non-nullable.
Applying changed schema text, including comments or formatting, records the new
source in one graph commit even when no table changes are needed. Applying the
same accepted source again leaves the graph unchanged.
Changes such as adding a required property to existing entities (add it nullable,
then promote it), changing a property
type (except enum widening and making a property required), changing edge endpoints, raising an edge's `@card` minimum or lowering its maximum, adding an
edge type with a non-zero `@card` minimum from an existing node type, changing a
node's implemented interfaces, adding a constraint other than `@index` and a node's `@unique`,
removing a constraint from a property that stays, and adding, removing or
changing an existing type's `@key` are rejected. A drop is rejected when it
would remove a constraint that also names a field that stays, such as
`@unique(a, b)` when only `a` is dropped. The plan reports the exact
unsupported step before anything changes.
A drop removes the declaration from the current schema and reclaims no storage
at apply. Older commits still read the dropped data until `omnigraph cleanup`
stops retaining them. Re-adding a property under a dropped name starts a new
property, so a historical query never reads the dropped values through it; see
[historical reads](https://www.omnigraph.dev/docs/branching.md#historical-reads). A dropped type's data is reclaimed by that cleanup. A
dropped property's values also stay in the table's current data files, beside
the properties that remain, until the next `omnigraph optimize`: every
optimize rewrites each fragment of a table that still stores a dropped
property's values, copying only the remaining properties. Optimize rewrites
main only, and cleanup deletes the old files only once nothing it retains
references them: a live branch keeps the commit it was created from and its
head, a tag keeps the snapshot it names, and the retention policy keeps older
commits. Merging a branch releases none of its files; only deleting the branch
does. To erase dropped data, run optimize after the drop, delete every branch
created from a commit before that optimize and every tag naming such a commit,
then run cleanup with a retention that excludes the commits before the
optimize, for example `--keep 1`; after that, the dropped data cannot be
recovered. Review the plan carefully, and treat that cleanup as the step that
cannot be undone. See [optimize](https://www.omnigraph.dev/docs/operations/maintenance.md#optimize) and
[cleanup](https://www.omnigraph.dev/docs/operations/maintenance.md#cleanup).
Cluster-managed graphs change schema through `omnigraph cluster apply`. Direct
schema apply and the server schema-apply endpoint refuse cluster-managed graphs.
## Diagnostic codes
Migration rejections may include a stable `OG-...` code. Match automation on the
code rather than the message text.
| Code | Meaning |
| ----------- | -------------------------------------------------------------------------------------------- |
| `OG-DS-102` | Drop a node type that contains entities. |
| `OG-DS-103` | Drop an edge type that contains entities. |
| `OG-DS-104` | Drop a populated property. |
| `OG-MF-103` | Add a required property to a populated type. |
| `OG-MF-104` | Make a property required while a stored row holds a null (reported by `schema apply`). |
| `OG-MF-106` | Change a property's type, including enum narrowing or renaming. |
| `OG-MF-107` | Add an edge type with a non-zero `@card` minimum from an existing node type. |
| `OG-MF-108` | Add `@unique` to a node type while two stored rows share a key (reported by `schema apply`). |
`omnigraph schema plan` includes these codes in human and JSON output, except
`OG-MF-104` and `OG-MF-108`, which only `schema apply` can know from the stored
rows.