Document Store (document)

Category: data | Form: | Symbol: Do

Store and query semi-structured documents

Document store for semi-structured JSON data. Supports insert, upsert, query, and find operations. Documents are stored as JSONB in PostgreSQL. Attach to action elements to inject the connection as an environment variable.

Guide

Store and query semi-structured documents

What It Does

Document Store is a data element that holds a collection of markdown pages in a per-element, git-backed Content-Addressable Storage (CAS) repository. Each page is a .md file addressed by a path (e.g. intro, guides/getting-started) and every write is committed, so pages carry a full version history retrievable through the page-history operation.

Beyond plain create/read/update/delete, the element offers hybrid full-text search (70% vector similarity, 30% BM25 keyword matching), server-side markdown-to-HTML rendering, bulk import/export, and an async vectorize operation that chunks pages by heading and generates embeddings via the LLM gateway to power semantic search.

As a data atom it has no wired ports — it is a leaf addressed entirely through its ops API. Action elements (python, javascript, rust-fn, …) reference a document through their uses list, and the runtime injects the document’s connection info as an environment variable inside the action’s handler.

Element Definition

Property	Value
Type	document
Category	data
Form	atom
Symbol	Do
Icon	description / #F59E0B
Storage backend	postgres
Wirable	false
LLM role	data

Properties

Field	Type	Required	Constraint	Description
description	string	No	maxLength 512	Human-readable description of what this document stores

States

State	Meaning
provisioned	Initial state after creation
active	Store is provisioned and ready
error	Store entered an error condition

Attaches

Modifier elements that can attach to a document: auth-policy, rate-limit, variable.

Capabilities

Capability	Description
git-backed-storage	Pages stored as .md files in a per-element CAS git repository
full-text-search	Hybrid full-text search using Tantivy (BM25) + vector embeddings (70/30 weight)
version-history	Per-page version history via git commit log, accessible through the page-history op
markdown-rendering	Server-side render of pages to HTML via the render op
vectorize	Async embedding generation for all pages, enabling semantic search

Error Codes

Code	Class	Retryable	Description
DOCUMENT_PAGE_NOT_FOUND	not_found	No	The requested page path does not exist in this document
DOCUMENT_STORAGE_ERROR	internal	Yes	CAS git repository operation failed (commit, read, or write)
DOCUMENT_VECTORIZE_FAILED	internal	Yes	Embedding generation failed during vectorize operation
DOCUMENT_INVALID_PATH	validation	No	Page path contains invalid characters or path-traversal sequences

Operations

All operations are addressed under the flat element URL /api/{circle}/{slug}/ops/<op-name>. The method and path below are from the op definitions.

create-page

POST pages · auth: write

Creates a markdown page in the document’s git-backed repository. Provide a path (e.g. guides/intro) and content (markdown string); the page is stored as a .md file and committed to CAS. Required input: path, content. Optional title (extracted from the first heading if omitted). Returns path and created.

get-page

GET pages/{page_path} · auth: read

Returns the full markdown content of a page at the given path. Returns 404 if the page does not exist. Output: path, content, title.

update-page

PUT pages/{page_path} · auth: write

Replaces the content of an existing page (the page must already exist) and creates a new git commit. Required input: content. Returns path and updated.

delete-page

DELETE pages/{page_path} · auth: write

Deletes a page. Returns deleted. (Surfaced in the danger UI section.)

list-pages

GET pages · auth: read

Returns a hierarchically sorted tree of all pages, including paths and titles. Output: pages[] of { path, title }.

move-page

POST move · auth: write

Moves or renames a page. Required input: from, to. Returns moved.

render

POST render · auth: read

Converts a markdown page to HTML, supporting standard markdown plus extensions (tables, code highlighting, math, diagrams). Provide either path (a page to render) or content (raw markdown). Output: html.

search

POST search · auth: read

Full-text search across all pages using hybrid search (70% vector similarity, 30% BM25 keyword matching). Returns matching pages with relevance scores and highlighted snippets. Required input: query; optional limit (default 10). Output: results[] of { path, title, snippet, score }.

import

POST import · auth: write

Imports pages from an external format. Input: format (markdown | html | text, default markdown), content, and target path. Output: imported, pages_created.

export

GET export · auth: read

Exports all pages as a single bundle, useful for backup or migration. Output: pages[].

page-history

GET pages/{page_path}/history · auth: read

Returns the git commit log for a specific page — all past versions with timestamps and change summaries. Output: commits[] of { sha, message, timestamp }.

vectorize

POST vectorize · auth: admin · async

Chunks all pages by heading, generates vector embeddings via the LLM gateway, and stores them for hybrid search. This is an async operation that may take time for large documents. Output: pages_processed, chunks_created.

write

POST write · auth: write

Convenience operation that creates a page if it does not exist or updates it if it does — equivalent to create-page for new pages and update-page for existing ones. Required input: path, content. Returns path and created.

Quick Start

Create the document

POST /api/{circle}/{project}/
Content-Type: application/json

{
  "element_type": "document",
  "slug": "handbook",
  "name": "Team Handbook",
  "spec": {
    "description": "Internal team handbook and runbooks"
  }
}

Write a page

POST /api/{circle}/handbook/ops/create-page
Content-Type: application/json

{
  "path": "guides/onboarding",
  "content": "# Onboarding\n\nWelcome to the team.",
  "title": "Onboarding"
}

Search across pages

POST /api/{circle}/handbook/ops/search
Content-Type: application/json

{
  "query": "onboarding checklist",
  "limit": 5
}

Render a page to HTML

POST /api/{circle}/handbook/ops/render
Content-Type: application/json

{
  "path": "guides/onboarding"
}

Common Mistakes

Updating a page that does not exist. update-page requires the page to already exist; on a missing path it returns DOCUMENT_PAGE_NOT_FOUND. Check the path with list-pages first, or use write, which creates-or-updates.

Using invalid page paths. Page paths are restricted to alphanumeric characters, hyphens, underscores, and forward slashes — no .. path-traversal sequences. A bad path returns DOCUMENT_INVALID_PATH.

Searching before vectorizing. Hybrid search blends vector similarity with keyword matching. Run the async vectorize operation so embeddings exist before relying on semantic relevance in search results.

Over-long description. The description property is capped at 512 characters; longer values fail element creation/update validation (description_max_length).

Relationships

• Attaches to: auth-policy, rate-limit, variable

Capabilities

• git-backed-storage: Pages stored as .md files in a per-element CAS git repository
• full-text-search: Hybrid full-text search using Tantivy (BM25) + vector embeddings (70/30 weight)
• version-history: Per-page version history via git commit log, accessible through page-history op
• markdown-rendering: Server-side render of pages to HTML via the render op
• vectorize: Async embedding generation for all pages, enabling semantic search

Properties

Property	Type	Default	Description
description	string	—	Human-readable description of what this document stores

Operations

activity

Get /ops/activity | Auth: Read

Get activity events for this element

Scope depends on element capabilities: individual elements query by element_id, project-form elements with activity-scope-members include member activities, circle-level elements with activity-scope-all query the entire circle. Gracefully returns empty list if activities table is missing (old circles).

attachments

Get /ops/attachments | Auth: Read

List all modifiers and resources attached to this element

Returns both modifiers (policy enforcement) and resources (data injection) with is_modifier flag to distinguish. Items in the generated MODIFIER_TYPES list are modifiers; everything else is a resource. Includes cascade_policy and version pin info.

batch_stats

Get /ops/batch_stats | Auth: Read

Get per-element statistics for all children of this element

Returns per-child stats plus an aggregate. Most meaningful on compound or manifest form elements (repositories, circles, projects); atoms have no children so the result is an empty children array with a zeroed aggregate. Uses efficient GROUP BY SQL. Weighted averages for eval scores.

compose

Post /ops/compose | Auth: Execute

Batch add and remove modifiers on this element in a single call

Declarative composition: add modifiers by ref path (slug or path@version) and remove by attachment ID, all in one atomic call on the target element. Each ‘add’ entry resolves the source element, validates topology, attaches with optional priority and cascade policy. Each ‘remove’ entry deletes the attachment row. Returns a summary of what was added and removed. Example: compose({ add: [{ref: “my-prompt”}, {ref: “rate-limit/api@v2”, priority: 50}], remove: [{attachment_id: “uuid”}] })

context

Get /ops/context | Auth: Read

Get connected elements (graph traversal)

Graph traversal showing all connected elements with their relationship type (contains, contained_by, references, referenced_by, attaches, etc.). Use ?depth=N to control traversal depth (default 1) and ?types=actor,data to filter by element types.

create

Post /ops/create | Auth: Write

Create child element

POST to the parent path — element_type goes in the request body, NOT the URL. Both element_type and slug are required and must be non-empty. Name is derived from slug if omitted. Writes to both Git and PostgreSQL. All elements are stored flat under the circle — no intermediate library wrapper rows.

create-page

Post /ops/pages | Auth: Write

Create a new page in the document

Creates a markdown page in the document’s git-backed repository. Provide a path (e.g., “guides/intro”) and content (markdown string). The page is stored as a .md file and committed to CAS.

delete

Delete /ops/delete | Auth: Admin

Delete element (soft delete)

Soft delete — sets state to ‘deleted’ but retains the record. Cannot delete elements that have children (has_no_bond precondition) or active runs. Requires admin auth and confirmation.

delete-page

Delete /ops/pages/{page_path} | Auth: Write

Delete a page

disable

Post /ops/disable | Auth: Admin

Disable element (hides and prevents use)

Idempotent — safe to call on already-disabled elements. Optionally pass a reason string. Disabled elements cannot be invoked or executed. Inverse of enable.

enable

Post /ops/enable | Auth: Admin

Enable element (makes usable and visible)

Idempotent — safe to call on already-enabled elements. Transitions element to ready/enabled state. Cannot enable deleted elements. Inverse of disable.

export

Get /ops/export | Auth: Read

Export the entire document

Exports all pages as a single bundle. Useful for backup or migration.

export_bundle

Get /ops/export/bundle | Auth: Read

Export element as downloadable git bundle

On non-root-namespace elements, returns a binary git bundle. On root-namespace (circle) elements, dispatch hands off to the circle’s own export_bundle op, which returns a multi-element JSON envelope with one base64 bundle per child element — this is intentional, not an error.

get

Get /ops/get | Auth: Read

Get element details

Element is already resolved by the routing layer — this returns the cached element, not a fresh DB query. Use the path /api/{circle}/{slug} to address elements.

get-page

Get /ops/pages/{page_path} | Auth: Read

Get a page by path

Returns the full markdown content of a page at the given path. Returns 404 if the page does not exist.

import

Post /ops/import | Auth: Write

Import pages from external format

import_bundle

Post /ops/import/bundle | Auth: Write

Import git bundle into element

Accepts a base64-encoded git bundle in the JSON bundle_base64 field. Use overwrite=true to replace existing elements with same slug (default skips duplicates). Imported elements get new UUIDs. Returns counts of imported/skipped elements and any errors.

intention

Get /ops/intention | Auth: Read

Get element intention with full inheritance chain

Returns three levels: direct (this element’s intention), inherited (from category and root), and resolved (final merged intention). Useful for understanding an element’s purpose in context of its hierarchy.

list-pages

Get /ops/pages | Auth: Read

List all pages in the document

Returns a tree of all pages in the document, including paths, titles, and modification timestamps. Pages are sorted hierarchically.

move-page

Post /ops/move | Auth: Write

Move or rename a page

page-history

Get /ops/pages/{page_path}/history | Auth: Read

Get version history for a page

Returns the git commit log for a specific page, showing all past versions with timestamps and change summaries.

promote

Post /ops/promote | Auth: Admin

Promote element configuration to a target environment

Only for manifest-form elements (projects). Environments advance: dev → demo → live. dev→demo requires member+ role, demo→live requires admin. Freezes member versions at promotion time (creates snapshot). Persists environment config to spec.environments.

readme

Get /ops/readme | Auth: Read

Get element README.md content

Reads README.md from the element’s git repository. Returns empty content (not an error) if no README exists. Always returns markdown format.

readme_update

Post /ops/readme_update | Auth: Write

Update element README.md content

Creates or overwrites README.md in the element’s git repo. Commits to the draft branch. Content must be provided as a markdown string.

remove-modifier

Post /ops/remove-modifier | Auth: Execute

Remove an attached modifier from this element by attachment ID

Removes a modifier/resource attachment by its row ID. The ID comes from the attachments or context API. This is the reverse of attach — called on the target element, not the source.

render

Post /ops/render | Auth: Read

Render a page to HTML

Converts a markdown page to HTML. Supports standard markdown plus extensions (tables, code highlighting, math, diagrams).

restore

Post /ops/restore | Auth: Admin

Restore element to a specific version

Automatically snapshots the current state before restoring (creates a ‘Before restore to vN’ version entry). Writes restored spec to git as .triform/spec.yaml. Git failures warn but don’t fail the operation — DB state is authoritative. Cannot restore deleted elements.

schema

Get /ops/schema | Auth: Read

Get element input/output schema (MCP tools/list compatible)

Returns type-level port schemas from the TypeRegistry — not instance-specific overrides. Includes direction (input/output), required flag, and JSON schema per port. Useful for understanding what data an element accepts and produces.

search

Post /ops/search | Auth: Read

Full-text search across all pages

Searches page content using hybrid search (70% vector similarity, 30% BM25 keyword matching). Returns matching pages with relevance scores and highlighted snippets.

source

Get /ops/source | Auth: Read

Get any file’s content from the element’s git repository

Reads an arbitrary file from the element’s CAS-backed git tree by its relative path. Same store as readme, just generalized. Path safety: rejects .. traversal, leading /, and null bytes. Use this to view main.py for action elements, asset files for SPAs, etc. Returns empty content (not an error) if the file doesn’t exist.

source_branches

Get /ops/source/branches | Auth: Read

List Source branches for this element

Returns the standard draft/demo/live Source branches, their current commits, and promotion relationships. Use GET /api/{element_path}/ops/source/branches.

source_fixtures

Post /ops/source/fixtures | Auth: Write

Dry-run or apply approved Source seed fixtures

Scans .triform/fixtures/ manifests from the addressed data element Source repo. Defaults to dry_run=true and never imports live runtime data. Apply requires dry_run=false plus confirm=true and dispatches approved records through existing generated element ops.

source_promote

Post /ops/source/promote | Auth: Write

Promote Source branch forward

Promotes draft to demo or demo to live through the generated element op path. Direct Git pushes to demo/live are blocked by Source policy.

source_repair

Post /ops/source/repair | Auth: Write

Inspect or repair the element Source index

Runs Source repair through the element operation path. Defaults to dry_run=true; set dry_run=false only after reviewing a dry-run report.

source_status

Get /ops/source/status | Auth: Read

Get Source control status for this element

Returns the branch-aware clone URL, checkout commands, current draft commit, child source-link count, portable export summary, Source health, warnings, and auth hints for the addressed element. Use the element-first path: GET /api/{element_path}/ops/source/status.

source_validate

Post /ops/source/validate | Auth: Read

Validate Source branch contents

Validates a Source branch before accepting local Git workflow changes or promotion. Defaults to branch=draft and rejects runtime data, generated output, secret material, and unreadable CAS refs.

stats

Get /ops/stats | Auth: Read

Get aggregate statistics for this element

Health status is computed: error if errors_per_day > 5 or success_rate < 0.8, warning if errors_per_day > 0 or success_rate < 0.95. Firing alerts escalate health to error/warning. Default period is ‘day’. Returns runs_per_day, success_rate, avg_duration_ms, and more.

tree

Get /ops/tree | Auth: Read

Get the element’s position in the graph — ancestors, children, references, and subtree statistics

Uses per-circle ElementGraph cache for O(1) lookups. Returns ancestors (containment chain), children (direct), members (references), referenced_by (reverse refs), attachments, and subtree stats. Default depth is 3, max is 10. Pass ?include_metadata=true for name/state on each node.

update

Patch /ops/update | Auth: Write

Update element

Partial update — send only the fields you want to change. spec, name, and intention are all independently optional. spec MUST be a JSON object when present; deep-merged into the existing spec by default. Empty {"spec":{}} preserves existing spec content but still records a new version (no-op for content, not for version state). To clear/replace the entire spec wholesale send {"spec":{...},"deep":false}. List-typed spec fields use replace semantics (the patch list replaces the existing list, no array merging). Coordinates Git + DB writes. Slug cannot be changed after creation.

update-page

Put /ops/pages/{page_path} | Auth: Write

Update an existing page

Replaces the content of an existing page. The page must already exist. Creates a new git commit in the CAS repository.

update_meta

Patch /ops/update_meta | Auth: Write

Update element metadata (lightweight merge — does NOT bump version or snapshot spec)

Shallow JSONB merge into element.meta. Top-level keys in the provided value replace existing meta values; other keys are preserved. Used for UI metadata like canvas positions, panel state, viewer preferences. Wire-shape op_name is update_meta (distinct from update) so SSE subscribers + the cache auto-invalidator can distinguish lightweight metadata changes from spec edits without inspecting the payload. The MutatingElementStore wrapper stamps this op_name on the lifecycle event emitted by update_element_meta storage calls.

vectorize

Post /ops/vectorize | Auth: Admin

Generate vector embeddings for all pages

Chunks all pages by heading, generates vector embeddings using the LLM gateway, and stores them for hybrid search. This is an async operation that may take time for large documents.

version

Get /ops/version | Auth: Read

Get current version or full history

Returns current version by default. Pass ?history=true for full version history (up to ?limit=N, default 50). Versions are backed by the element_versions table. Every spec update creates a new version entry.

write

Post /ops/write | Auth: Write

Write content to a page (create or update)

Convenience operation that creates a page if it doesn’t exist or updates it if it does. Equivalent to create-page for new pages and update-page for existing ones.

Error Codes

Code	Class	Retryable	Description
DOCUMENT_PAGE_NOT_FOUND	not_found	no	The requested page path does not exist in this document
DOCUMENT_STORAGE_ERROR	internal	yes	CAS git repository operation failed (commit, read, or write)
DOCUMENT_VECTORIZE_FAILED	internal	yes	Embedding generation failed during vectorize operation
DOCUMENT_INVALID_PATH	validation	no	Page path contains invalid characters or path-traversal sequences