Home

Deep dive 04 · Architecture

One backend, many ways in.

The system keeps capture flexible while preserving one identity, permission, storage, and retrieval model.

System map

Every client reaches the same source of truth.

Ten edge functions handle focused jobs around a shared Supabase core. New clients reuse the same product model instead of creating a parallel library.

Archi.tldraw · System map
Clients reuse focused edge functions around one permission, storage, and retrieval model.

Surfaces

Eleven ways in, and not all of them are shipped.

Built and shipped are different claims, so the status column says which. Nothing here is in a store: the Safari extension is signed and notarized for direct install, and everything else is either live on the web or handed out by hand.

Public URLs

One reference, packaged for people outside the workspace.

The public view keeps attribution, platform, recency, and the actual screen together, so a share link has context instead of becoming another loose screenshot.

AgentUX public URL preview showing a Wealthsimple reference card with metadata, screenshot, collaborator avatars, and cursor markers
  • Web appLive

    The main surface: browse, capture, label, share. Seven routes.

  • Telegram botLive

    Send a screenshot or a link from your phone and it lands in the catalogue without opening the app.

  • Safari extensionNotarized, direct install

    Right-click any image to save it while you are still reading the page. Signed and notarized, installed from a private page inside the app; a Mac App Store listing is more process than an internal tool earns.

  • Chromium extensionBuilt, sideloaded

    The same capture, built for Chrome and Edge. Loaded unpacked, and not submitted anywhere.

  • iOS appTestFlight

    Native client at V0.1, distributed through TestFlight.

  • Folder uploadLive

    Drop a whole directory in. A single dropped folder name pre-fills the flow it belongs to.

  • iOS ShortcutLive, dormant client

    Share-sheet capture. The endpoint stays live for shortcuts already in use; the native app replaced its setup screen.

  • REST APILive

    Read access for products and scripts. Search, or fetch one reference.

  • MCP serverLive

    Eleven tools for agents. Nine read, two write, and both writes land in a human queue.

  • Public URLsLive

    Send a focused collection to someone without an account, instead of a folder of screenshots.

  • Prototype hostingLive

    Upload an HTML prototype and it serves on its own subdomain.

Edge layer

Ten functions, each with one job.

Grouped by what they are for. The three metadata functions run after the fact, on first render rather than at capture, so a slow third-party lookup never blocks someone saving a reference.

Getting references in

  • telegram-botsync

    Routes an inbound message: an image becomes a screenshot, an X status becomes a video reference, any other URL becomes a link.

  • inbox-ingestsync

    Accepts one image from an agent and lands it unlabelled in a holding group for a person to triage.

  • shortcut-uploadsync

    The same routing as the bot, for the iOS share sheet.

Labelling

  • label-ingestsync

    The only surface that writes a label. Lists the oldest unlabelled screens, and saves one label. It will not overwrite a label that arrived while it was reading.

Access

  • auth-loginsync

    Validates an email and passcode, issues a session. Repeated failures lock out temporarily.

  • auth-adminsync

    Members, roles, and agent session tokens: mint, rotate, disable, force-logout. Minted credentials are shown once.

Filling in metadata after the fact

  • fetch-tweet-metadatadeferred

    Author, excerpt, poster and like count for a batch of tweets. Runs on first render, not at capture.

  • fetch-youtube-metadatadeferred

    Title, channel and thumbnail through a public endpoint, no API key.

  • fetch-link-metadatadeferred

    Title, description and image for saved links. Results are not persisted, and private hosts are refused.

Housekeeping

  • purge-orphan-storagesync

    Deletes files nothing in the database points at. Operator-invoked, and safe to re-run.

One capture, traced

A screenshot from a group chat to a searchable reference.

The rest of this page describes the shape of the system. This follows one thing through it. Telegram is a capture path because the designers were already dropping screenshots into a chat, so the habit got supported instead of fought.

  1. Send

    A designer sends a screenshot to the bot.

  2. Authenticate

    Webhook and teammate checks fail closed.

  3. Store

    Media is uploaded and an unlabelled row is created.

  4. Search

    The bot replies. The draft is searchable by name.

  5. Verify

    A person reviews it before agents see it.

Telegram handles capture. Human review remains the publishing boundary.

Why “verified” can be trusted

Labelling is never automatic. There is no server-side model call anywhere in the project: an AI drafts a label through a prompt a person runs, or an agent submits one over MCP. Either way it arrives as “needs review”. Four independent things stop a machine publishing to the library.

  1. The write endpoint hard-codes the status to needs_review. A submitted "verified" is discarded, not honoured.
  2. Verified is written in exactly one place in the codebase: a person clicking Verify, behind ten validation rules.
  3. A human pasting AI output that claims verified gets a draft instead, with a visible warning.
  4. The read API distrusts it anyway, filtering AI-authored records that are not verified on both search and direct lookup.

What happens when it goes wrong

Sender is not on the team
Ignored silently. No reply at all, so the bot does not confirm it exists.
Unsupported file
A help reply naming what is accepted. Nothing is stored. A video or PDF sent as a document is refused.
Upload fails
Reported in the chat. No row is created, so there is no orphan record.
Insert fails after the upload succeeded
Reported in the chat, and the file is left orphaned in storage. This is the case the manual purge function exists to clean up.
The same URL again
Answered as "already in catalogue" rather than as an error.

Offline capture

A lost connection should not lose the reference.

Mutations wait in a local queue while image blobs remain available. Reconnection flushes the queue in order.

Archi.tldraw · Offline sync
Local persistence protects the reference; ordered replay protects the final state.

Security model

Humans and machines use different credentials.

The product separates durable human authentication from narrow machine sessions. Each credential follows the risk of its client.

CredentialStorageControl
Human passcodeArgon2id hashAccount lifecycle
Agent sessionSHA-256 hashScoped, expiring, revocable
Telegram webhookAuthenticity checkReject untrusted requests

Platform details

Small infrastructure choices protect the experience.

Prototype proxy

External prototypes load through a controlled hosting layer.

Share target

The service worker intercepts only the share flow it owns.

Email-keyed collaboration

Team identity stays stable across invitations and comments.