API Routes As Source Of Truth

The web app owns backend route contracts for every surface. Desktop, mobile, Chrome, and future surfaces should call the same /api/v1/* endpoints through @overlay/api-client rather than copying route-specific fetch logic.

​

Route catalogs

The filesystem under src/app/api/**/route.ts is authoritative for route existence and exported HTTP methods. Use the full web API route catalog for behavior, authentication boundaries, and implementation files, or the compact API route catalog for a one-line inventory. npm run docs:health verifies that both catalogs match every route entry point. The generated OpenAPI document intentionally covers a narrower public contract set.

​

Rules

• Add or update endpoint contracts in @overlay/app-core.
• Add transport wrappers in @overlay/api-client.
• Keep route behavior backward compatible unless a real bug is found.
• Prefer parsed typed methods when callers only need data.
• Add *Response methods when callers need status codes, streaming bodies, redirects, or custom error handling.
• Stream chat directly through /api/v1/conversations/act. Disconnect recovery comes from persisted conversation state; /api/v1/conversations/stream-auth and an external replay relay are not public contracts.

​

Bootstrap

/api/v1/bootstrap returns the canonical serializable frontend registry:

• brand config
• navigation
• settings sections and panels
• feature flags
• feature modules
• tool and integration registry metadata
• model provider metadata
• policy gates
• theme metadata
• models, user, entitlements, defaults, and UI settings

Surfaces should use bootstrap first, then render their native shell from those registries. GET /api/v1/model-catalog returns the authorized live AI Gateway catalog for language, image, and video models. Clients register only media entries whose Gateway pricing shape is supported by Overlay’s reservation calculator; unpriced media models stay unavailable instead of creating unreconcilable usage. The settings model catalog uses this route for chat enablement and image/video defaults.

​

Agent environments

/api/v1/agent-environments/** is the only public control plane for connected hosts. Browser clients use AgentEnvironmentsClient for list, enrollment creation, approval, root-scope changes, managed Overlay Cloud provisioning, and revocation. The portable host uses the bridge protocol for enrollment, credential issuance and refresh, heartbeat, capability refresh, command polling/acknowledgement, and event batches. Browser mutations require the signed-in workspace owner or admin. Host routes accept no browser cookie authority: a redeemed one-time code establishes a pending environment, initial credentials require an Ed25519 proof, and every later request is bound to the exact environment, workspace, audience, allowed method, expiry, request body, timestamp, and single-use nonce. Raw enrollment codes, proof challenges, credentials, and request nonces are never persisted. Enrollment-session creation opts out of BFF response idempotency because its response contains the only copy of the code; retrying creates a new short-lived code instead of persisting that secret in the idempotency store. POST /api/v1/agent-environments/managed provisions an Overlay-managed environment through the private @overlay/sandbox-runtime contract. The response contains only the environment, public lease status, and the /workspace approval hint. Provider selection, raw diagnostics, provider references, credentials, image details, and billing remain server/operator concerns. Managed provisioning deliberately redeems the normal one-time enrollment inside the sandbox and waits for the same browser approval; it is not a second host authentication path. POST /api/v1/sandbox/run is the provider-neutral generated-output execution route and is supported by both application-data providers. Its thin route wrapper delegates provider work, metering, and artifact persistence to the server boundary; provider SDKs never enter src/app. GET, PUT, and DELETE /api/v1/agent-bindings list, upsert, and disable the workspace-owned mapping from an Overlay agent to one approved environment, advertised ACP adapter, and granted working directory. A bound agent follows the ordinary POST /api/v1/conversations/message mention path; no second remote-chat endpoint or transcript exists. POST /api/v1/conversations/run/remote is the narrow browser control for Cancel, Retry, Resume, and Start fresh. POST /api/v1/conversations/run/remote/request resolves a pending permission or structured elicitation only after rechecking the human’s current workspace and room authority. GET /api/v1/conversations/run/remote/artifacts/:artifactId authorizes the current participant before redirecting to a short-lived object-store URL. Hosts create and complete scoped artifact uploads through the environment’s artifacts subresource; only checksum-, type-, size-, malware-, tenancy-, and retention-validated objects may be projected into the transcript. These controls are authenticated and workspace-scoped through @overlay/api-client. The internal-secret-only POST /api/v1/agent-environments/artifacts/cleanup endpoint lets the Convex scheduler request object deletion from the BFF without moving object-store credentials into Convex. The same internal-secret boundary protects POST /api/v1/agent-environments/operations/reconcile: the Convex scheduler keeps run supervision durable while the BFF performs provider-credentialed sandbox settlement retries. PostgreSQL runs the equivalent work through its application-data maintenance scheduler.

​

Cross-Surface Adoption

Desktop and Chrome can use the React DOM packages directly where practical. Mobile should use @overlay/app-core, @overlay/api-client, and pure controllers from @overlay/app-core/modules, while keeping React Native rendering in mobile-specific components.

​

Workspace Memory

GET /api/v1/memory is workspace-scoped. It returns all memories in the authorized active workspace by default and accepts memberPrincipalId to filter by one active human member. Each row exposes the creator’s workspace identity and whether the authenticated member may delete it. Human POST requests write both the active workspaceId and authenticated creator userId; an authorized agent turn instead uses the opaque agent-memory:<agentId> owner. Update and delete remain creator-only. Persistent chat context and hybrid retrieval both resolve memories by workspaceId, so memories created by one member or agent are available as shared context to other members of that workspace. Files remain owner-scoped during hybrid retrieval. Provider repositories must preserve the distinction: workspaceId owns memory context, while the legacy-named userId field stores an opaque human-or-agent owner for attribution and mutation ownership. PostgreSQL therefore must not foreign-key memory or memory-index ownership to the human users table; application cleanup plus the users_cleanup_human_memory_owner compatibility trigger preserve human account deletion across rolling upgrades and rollback.

​

Collaboration message contracts

• Personal and organization workspaces expose the same collaboration routes. Personal lifecycle mutations reject archive, ownership transfer, promotion of another owner, and removal, suspension, or demotion of the creating owner. Other Personal members may be invited, managed, and removed normally.
• POST /api/v1/conversations/message returns the persisted messageId. Agent-invocation callers use that ID directly; transcript refetch state is not an identity source.
• POST /api/v1/conversations/message accepts an optional memoryEnabled turn preference. Saving an agent-triggering room message opens the durable agent run server-side; the retired browser-held /api/v1/conversations/agent-reply path is not a public contract. Hosted and connected turns load bounded workspace memory and room context only when the preference is enabled. The provider-neutral collaboration repository schedules validated human- and agent-message extraction, while Convex and PostgreSQL enforce the same room, author, owner, and idempotency rules. Extraction-model context is separately bounded to messages from the target human or exact agent principal; other room participants are not sent to that additional provider call.
• Serialized collaboration messages may include editedAt and editHistory: Array<{ content, editedAt }>. Both Convex and Postgres append the replaced body before an edit.
• PATCH /api/v1/conversations/notifications accepts either notificationIds or conversationId. Opening a DM or channel uses conversationId so only that room’s unread activity is cleared.
• Convex-backed browsers use native watchConversationListVersion and watchNotifications subscriptions. /api/v1/conversations/events is reserved for the Postgres provider and must not be mounted as a Convex fallback.
• Personal chat creation is deferred until the first send. Newly created DMs and channels are client-side drafts until their first message succeeds: they are not inserted into the sidebar list, and navigating away removes the current member from the empty room (falling back to a self-archive when the room must retain a human participant). A successful or remotely observed first message commits the draft and emits the ordinary conversation-created event.

​

Personal Chat AgentRun contracts

Connected-agent public resources are reserved under /api/v1/agent-environments/** and /api/v1/agent-bindings; they remain unmounted while the default-off control-plane flags are disabled. Both database providers use ConnectedAgentRepository for durable command claims and ordered event acknowledgement. Provider-branded sandbox identifiers and reusable host credentials are never public contract fields.

• POST /api/v1/conversations/act streams Chat mode directly over SSE. For a persistent conversation, the server drains a second stream branch after a browser disconnect and performs one final assistant-message write; it does not persist token deltas or mirror the stream through Cloudflare.
• The same endpoint accepts personalChatMode: "work". It starts a durable WorkflowAgent and immediately returns the workflow’s default writable stream as an SSE UI message stream. The WorkflowAgent writes raw ModelCallStreamPart chunks to the run stream, which the route transforms through createModelCallToUIChunkTransform so the client receives token, tool, and approval updates in real time. Work mode closes the stream when the workflow finalizes or fails.
• Both Chat and Work streams preserve provider-native source-url and source-document UI parts. The transcript adapter owns compatibility with the legacy source shape and projects incomplete tool/reasoning part states to a terminal state once their response is completed, cancelled, interrupted, or failed.
• GET /api/v1/conversations/run?conversationId=<id> returns an active AgentRun when any model slot is non-terminal, otherwise the most recent terminal run. Clients use this response—not conversationMessages.status—to decide whether Stop or Send is available after refresh.
• POST /api/v1/conversations/stop atomically transitions active AgentRuns to cancelled, projects the interrupted state into their assistant messages, and best-effort aborts ToolLoop executions running in the same server process, and cancels attached Workflow runs. Assistant-message status is a display projection and is never consulted to decide whether Stop is active.
• POST /api/v1/conversations/run/approval accepts the active conversation, AgentRun and hook token plus an approve/deny decision. It rejects stale tokens and resumes only the current waiting_for_approval Work run.
• GET /api/v1/conversations/run/metrics?from=<epoch-ms>&to=<epoch-ms>&limit=<n> returns per-runner observations with explicit sample counts. It reports latency, provider cost, Workflow step/storage observations, disconnect and recovery outcomes, tool reliability, cancellation timing, and stale-run frequency. It deliberately returns no runner recommendation.
• POST /api/v1/conversations/run/metrics-event records the active browser’s disconnected or reconnected lifecycle observation for an exact AgentRun. It cannot reliably observe every abrupt network loss, so report denominators remain explicit.
• @overlay/api-client exposes conversations.currentRun() and conversations.stopResponse() plus conversations.submitRunApproval(), conversations.runMetrics(), and conversations.recordRunMetricEvent() for these contracts. Both Convex and Postgres repositories must preserve identical strict lifecycle transitions and metric merge behavior.
• The retired conversationMessageDeltas table, delta hydration queries, stream-auth route, and Cloudflare relay are not part of the API contract. Both runners persist one terminal assistant-message snapshot.

​

File listing

GET /api/v1/files?page=true&limit=<n>&cursor=<cursor> returns the standard { data, nextCursor, hasMore } envelope. @overlay/api-client exposes this through files.getPage(). Calls without page=true retain the historical array response, backed by the same bounded first page, so older web/desktop consumers remain compatible during rollout. Convex serves pages through files/files:listPage; the legacy files/files:list query is capped and exists only for compatibility and internal adapters that have not adopted cursors yet.

​

Database pagination (Phase B)

Convex list queries for conversations, projects, notes, and automations now accept limit and cursor parameters (beforeLastModified for conversations, beforeUpdatedAt for others). The default page size is 100 (down from unbounded/200/300). Over-fetch is bounded at 3x the page limit to account for in-memory filters. The BFF paginateArray helper still handles final client-facing pagination on the merged result. watchMessages now accepts an optional limit parameter. When provided, it watches only the most recent N messages (the “tail”) instead of the entire transcript. The client passes limit=200 by default; older history is loaded on demand via getRecentMessages.

​

One-time migrations

Legacy migration-on-read patterns (e.g. knowledge-base workspace binding) are being replaced with one-time migrations tracked by the migrationMarkers Convex table. Each migration is identified by a key (e.g. kb_personal_workspace_binding) and scoped to a user or workspace. The BFF checks platform/migrations:isComplete before running the migration and calls platform/migrations:markComplete after. Subsequent requests skip the migration entirely.

​

Rate limiting

The rate limit provider is selected at bootstrap. Redis is preferred when configured (TCP or Upstash REST), falling back to Convex. The Convex takeManyByServer mutation no longer prunes expired windows in the request path — pruning is handled by a periodic cron job (pruneExpiredWindowsInternal, every 5 minutes). This eliminates N extra deletes per authenticated BFF request.

​

Realtime and shared resource state (Phase C)

​

Versioned conversation list reconciliation

Personal conversation mutations (create, update, remove in convex/chat/conversations.ts) now emit conversationEvents rows, just like collaboration conversations. The conversationEvents table has a new by_userId_createdAt index for user-scoped version queries. watchPersonalConversationListVersion (in convex/chat/conversations.ts) returns the latest event _creationTime for a user. watchConversationListVersion (in convex/collaboration/directMessages.ts) returns the latest event for a workspace. The CollaborationRealtimeProvider subscribes to both and merges them via Math.max — any personal or collaboration change triggers a single version bump. When the version changes, the client fetches only the delta via GET /api/v1/conversations?updatedSince=<version-1000> and upserts individual rows into the chat list cache (upsertCachedChat / removeCachedChat). This replaces the previous full-list reload on every version change.

​

Presence subscription

watchPresence (in convex/collaboration/directMessages.ts) provides a Convex subscription for conversation presence and typing state. When Convex realtime is available, DirectMessageExperience skips the 15-second HTTP presence polling fallback. The heartbeat (45s) remains to keep the user’s own presence row alive. The query catches Unauthorized, WORKSPACE_ACCESS_DENIED, and CONVERSATION_ACCESS_DENIED errors gracefully (returning {ok: false, presence: []}) so that self-scoped conversation deletion/archive does not crash the React error boundary when the subscription is still active.

​

Knowledge source ingestion subscription

watchKnowledgeSourceStatus (in convex/knowledge/bases.ts) provides a Convex subscription for knowledge source status changes within a knowledge base. When Convex realtime is available, KnowledgeBaseWorkspace patches individual source statuses into local state instead of polling listSources every 2 seconds. The useVisibleReconciliation interval backs off from 15s to 120s when Convex is active.

​

Indexed mention and workspace search

Convex search indexes have been added to:

• conversations.search_title (searchField: title, filterFields: userId, workspaceId, deletedAt)
• files.search_name (searchField: name, filterFields: userId, workspaceId, deletedAt)
• notes.search_title (searchField: title, filterFields: userId, workspaceId, deletedAt)
• automations.search_name (searchField: name, filterFields: userId, workspaceId, deletedAt)
• skills.search_name (searchField: name, filterFields: userId, workspaceId)
• mcpServers.search_name (searchField: name, filterFields: userId, workspaceId)

search/mentions:searchMentions (in convex/search/mentions.ts) uses these indexes to return bounded top-10 results per category. GET /api/v1/mention-search?q=<query> is the BFF route that calls this query. The client searchMentions function tries the indexed endpoint first and falls back to scan-and-filter if unavailable. Knowledge bases and connectors remain client-side filtered (not in Convex search indexes). PostgreSQL route classification is explicit for release safety: connected-agent environments and bindings are supported. The provider-neutral workspace, agent, room, and conversation APIs are also enabled so an authenticated PostgreSQL browser can complete the same connected-agent vertical slice. Mention search is degraded to an empty bounded result, while sharing/search surfaces without a PostgreSQL repository and the Convex file-ingestion and Slack-import workers remain unavailable. PostgreSQL mode must not enter those Convex-only worker paths. searchWorkspaceChats (in convex/collaboration/channels.ts) now uses the conversations.search_title index for title matching instead of iterating all participant conversations. Message content search is still scan-based but limited to accessible conversations with a reduced message scan window (50 instead of 200).

​

Agent and workflow efficiency (Phase D)

​

Summary + unsummarized tail context loading

ActContextService.buildMessagesForModel now loads the conversation context summary first (via getContextSummary), then loads only messages at or after summarizedThroughCreatedAt using the new getMessagesSince Convex query. This replaces the previous behavior of loading the full conversation history via getMessages every turn and then compacting it. The summary is injected by compactMessagesForContext as before, but the initial load is bounded to the unsummarized tail. New Convex query: chat/conversations:getMessagesSince — accepts sinceCreatedAt and returns only messages at or after that timestamp. Falls back to full history when no cursor is provided.

​

Token-budgeted memory and skill context

Memory context is now bounded by MEMORY_CONTEXT_CHAR_BUDGET (2,000 chars / ~500 tokens). The top-10 memories by importance + recency are selected, then truncated to fit the budget. Skill context uses a lightweight directory (name + description only, no full instructions) bounded by SKILL_DIRECTORY_CHAR_BUDGET (1,500 chars / ~375 tokens). The agent loads full instructions on demand via the existing list_skills tool. New Convex queries:

• integrations/skills:listDirectory — returns _id, name, description, enabled only (no instructions)
• integrations/skills:getInstructions — returns full instructions for a single skill on demand

​

Durable document ingestion jobs

Document ingestion is now asynchronous via durable jobs instead of synchronous in the request path:

1. Client requests a presigned upload URL via POST /api/v1/files/presign
2. Client uploads the file directly to R2
3. Client creates an ingestion job via POST /api/v1/files/ingest-jobs with the r2Key + metadata
4. A cron-triggered Convex action (files/ingestion/runner:runMinuteTick) picks up queued jobs and calls the BFF processing endpoint
5. The BFF downloads from R2, extracts text, creates file records, and updates the job status
6. The client subscribes to job status via files/ingestion/jobs:watchIngestionJob or watchIngestionJobs Convex subscription

New Convex table: documentIngestionJobs with indexes by_userId_status, by_userId_createdAt, by_status_createdAt. New BFF routes:

• POST /api/v1/files/ingest-jobs — create a job after presigned upload
• GET /api/v1/files/ingest-jobs?jobId=<id> — get job status
• POST /api/v1/files/ingest-jobs/process — internal endpoint for the Convex runner

The synchronous POST /api/v1/files/ingest-document endpoint remains for backward compatibility (FormData upload flow).

​

Durable workflow step events

Workflow step events are now projected from the Workflow SDK into the Convex workflowStepEvents table by a cron-triggered action (automations/workflowEventProjector:runProjectionTick, every 10 seconds). Clients subscribe via automations/workflowEvents:watchWorkflowStepEvents or watchAutomationRunStepEvents Convex subscription for realtime updates. This replaces per-client SSE polling (every 2 seconds per client) with a single server-side poller. The SSE endpoint (GET /api/v1/automations/{runId}/events) remains as a fallback for non-Convex providers. New Convex table: workflowStepEvents with indexes by_workflowRunId_createdAt, by_userId_createdAt, by_automationRunId_createdAt.

​

Legacy cron removal

The legacy automation scheduler cron (automation_scheduler_legacy) has been removed. Durable automations are now always enabled and use the sleep()-based workflow (workflows/automation-schedule.ts) for scheduling. The OVERLAY_FEATURE_DURABLE_AUTOMATIONS feature flag has been removed.

​

Workspace Billing

Workspace billing is exposed only through authenticated /api/v1/workspaces/:workspaceId/billing* routes. The transport contract lives in @overlay/workspace-contracts and WorkspacesClient provides summary, initialize, subscription checkout, top-up, portal, and verification methods. Summary responses expose rollout eligibility, credit buckets, subscription status, and manager-only operational data but not the billing-account identifier. Route handlers resolve membership before invoking WorkspaceBillingService; only owners/admins can initialize, fund, subscribe, or open the portal. Clients never submit or select a billing-account identifier. Paid feature routes also never accept a billing-account identifier. The BFF resolves the authorized workspace and derives an API-key or resource subject; BillingPayerResolver chooses the account server-side. Convex hybrid search receives the already-resolved account only over the server-secret boundary, while Convex background indexing independently resolves the source workspace. The static owner-funded boundary check and billable-feature-coverage.test.ts are the source of truth for launch completeness. GET /api/v1/subscription and GET /api/v1/bootstrap expose the effective payer entitlement for the active workspace. An eligible organization workspace returns its workspace billing-account plan and remaining budget to every active member; personal workspaces and organizations outside the rollout retain personal entitlements. Clients send x-overlay-workspace-id on workspace-scoped entitlement reads so the model picker and the server-side charge path resolve the same payer. Billing settings, checkout, top-up, and portal mutations remain personal or workspace-manager operations and never become member-authorized merely because entitlement reads are shared.