Skip to content
Docs0.9GitHubInstall

Building on itApp protocol

Native app protocol

The app messaging transport serves one WebSocket endpoint, /v1, on a loopback address. A private proxy (for example tailscale serve) provides TLS. Every frame is a JSON text message. IDs are decimal strings. The message, topic, and other payload types are defined in crates/app-wire/src/lib.rs, which generates the web client’s web/src/kit/wire.ts and the iOS kit’s Wire.swift.

Plain HTTP GET and HEAD requests on the same port serve the built web client from [plugins.app] web (by default ~/.local/share/vendetta/web when it holds index.html), so the page and its WebSocket share an origin. Only regular files inside that folder are served. /media/… on the same port streams photos, videos, and audio, with or without a web folder.

The first client frame must arrive within 10 seconds and is one of:

Frame Reply
{"type":"pair","code":"…","device_name":"…"} {"type":"paired","device":"…","token":"…"}, then a snapshot
{"type":"hello","token":"…","epoch":"…","cursor":"…"} welcome then the missed events, or a snapshot

vendetta app pair creates a single-use code valid for 10 minutes; five wrong codes discard it. The token is shown once and stored by the server only as a SHA-256 digest. Failures reply {"type":"error","code":"unauthorized"|"pairing"|"busy"} and close. Ten failures in a minute make every new attempt wait. Revoked devices are disconnected within seconds.

A hello may list known, the content keys (same) of messages the client holds, newest first, at most 10,000: a snapshot then leaves out transcript history whose keys it all knows, since the client keeps its own copy. After a plugin restart, or a refresh, that is most of a snapshot.

The server sends each frame of 2 KiB or more, such as a snapshot or a batch, as a binary frame holding that frame’s JSON deflated (RFC 1951, no header), about a fifth of its size. Shorter frames are text.

snapshot carries epoch, seq, device, features (personal_hub: a plugin serves tasks, narration, narration_rewrite, transcription: the server can transcribe voice, which voice marks need, and voice_max_seconds: the most voice one send carries, a voice note or all of a marked send’s clips together; clients stop recording there and send; web_push_key, while Web Push is on; admin: this device’s person is the admin, whose devices manage people; master_project: the project the Master runs in, which clients list as no project, on home, the projects view, or a new agent’s choices, since the Master’s own row stands for it; absent without a Master), groups, topics, each chat’s 20 newest messages, older, root_read, queues, and progress (each chat’s Progress, absent when none has one); welcome carries the same epoch, seq, device, and features. Every later change is an event with a strictly increasing seq; a features {features} event replaces the features while connected, as when a person’s voice comes from the admin once their link is up, or goes when the admin stops giving it. Reconnect with the last epoch and seq: the server replays missed events, or sends a new snapshot when the epoch changed (plugin restart) or the bounded log no longer reaches the cursor. Clients merge snapshots into their own history by message ID; the server keeps messages in memory only, and photos, videos, and voice recordings as private copies. After the plugin restarts, each chat’s history is read again from its conversation’s transcript, an exited one’s too, so a device paired later sees it.

A snapshot’s older lists the chats whose older messages the server holds past those 20, by topic, "" for the root chat. older {request, topic?, before, limit?} reads them: up to limit (at most 100, else 50) of the chat’s messages ordered before message ID before, whether the server still holds that one or not, as older_result {request, topic?, before, messages, more}, oldest first, to that connection alone; more says it holds older ones still. A client asks when the owner scrolls up past the oldest message it holds, and when a snapshot’s oldest message in an older chat is newer than the newest it held before: it asks before that message until it reaches what it held or more is false, so a reconnect leaves no gap.

Several events sent at once, such as the missed ones, come as one events {seq,events} frame, which clients apply in one pass before taking its seq. An event that a later one replaces is left out of it: a message, topic, group, queue, read marker, or task list keeps its first event’s place, type, and seq with the content of its last, so a new message is still a message.

Events (type):

  • group {group:{id,name,shared_by?,machine?}}: a project. machine {name,state} for a container project; shared_by, the sharer’s display name, for a project another person shares, which is its own group, never merged into one of this person’s.
  • topic {topic:{id,title,state,group,master,manager,managers?,read,created_ms,avatar?,tools?,pins?,muted,kept,ephemeral,reload_pending,deletes_ms?,machine?,start?}}: created, renamed, given a picture, muted, unmuted, kept, or no longer kept. title is the agent’s name without status symbols, icons, or the project. state is running, idle, question, approval, starting, unknown, or exited; starting is an agent whose provider has reported nothing since its process started, a few minutes at most, as while a long conversation resumes. History entries and overview agents use the same states. avatar, the chat’s picture, holds exactly one of emoji, shown in place of the chat’s initials, or image, a blob ID fetch returns: a square JPEG, PNG, or WebP. tools, only while state is running, lists the tool calls the agent and its subagents have started and not yet finished, oldest first, each {name,detail?} as in ⏺ Bash(cargo test); an agent waiting on subagents that still work is running, its own turn over. With none open, it holds the newest call with done: true until another starts or the agent stops running, as a terminal keeps showing it while the agent thinks. A topic event follows each change. It is live state, never history, for each device to show or hide. manager marks an agent that manages a team, and managers names the chats of the managers whose teams the agent is in, oldest first, leaving out a manager with no chat, absent when none is left; a topic event follows each change. Clients show a member beneath each of its managers, marked there with the other managers’ names. pins lists the pages pinned to the chat in the order they were pinned, each {page,title,blob}: page is the page’s file name without .html, kept as the agent rewrites the page; title its <title> or readable file name; blob the ID fetch returns its current content for. muted, always present, marks a muted agent: its chat updates and counts unread as any other, but nothing about it notifies the owner (no push, client notification, or sound) except its alerts, and the app icon’s badge leaves it out. Agents a manager creates start muted, and the manager relays what the owner needs. A manager’s quiet turn (see the protocol) pushes nothing for its reply and progress either, in any chat; its questions and approvals push as usual. Clients show a muted glyph on the chat’s rows and header. kept, always present, marks a conversation the owner keeps: auto-delete never removes it. deletes_ms, on a stopped conversation only while auto-delete is on and would remove it, is when it will; clients show “Deletes in …” from it, and a bookmark glyph on kept chats’ rows and header. ephemeral, always present, marks a conversation that goes once idle for the server’s ephemeral_after_ms (see the protocol), whether or not auto-delete is on; its deletes_ms, live or stopped, is when it will while it stays idle, absent while it works, asks, or waits on an approval. Clients show a timer glyph on ephemeral chats’ rows and header. reload_pending, always present, is live state: the session reloads for new tools the admin shared once it is idle (it is working, asking, or waiting on approval now). Clients show a small quiet marker on the chat’s row and header (“Updating tools” where words are shown), never as a state of its own. declared {folder,id,paused,failed?}, only for an agent a project’s vendetta.toml declares: it always runs, a stop pauses it, and failed (crash_loop, start_uncertain) waits for the owner. Clients show a declared badge, and Start (agents start_declared {folder,id}) while paused or failed. machine {name,state}, only for an agent in a project machine, names it, with state running, stopped (its conversations are exited until resumed), unreachable (the state shown is the last one reported, and it takes no input until the machine is back), or unavailable (the container runtime cannot be used). Clients show a machine badge with its name on the chat’s rows and header, and the state when it is not running; a topic event follows each change. History entries and overview agents carry the same machine. shared {by,state}, only for a chat of a project another person shares (see Shares): by their display name, state linked or unreachable (the last state reported, and no input). A shared project is full access: the chat works as one’s own, but stopping and deleting it stay with the sharer, as does resuming one the sharer’s project file declares; its declared shows with folder empty. Clients show “Shared by NAME” on its rows and header and offer no Stop or Delete. History entries and overview agents carry the same shared. shares [{id,name}], on a chat of a project this person shares, lists who has it; clients show “Shared with …”. start {phase,reason?,since_ms,retry?,manager?}, only on a new agent’s chat until the agent is ready (see A new agent’s start).
  • topic_deleted {topic}.
  • message {message} and edit {message}: a full message replaces the old one.
  • progress {topic?, id, base, rev, from, responses, same, thinking?, links?, refs?, edited}: a running turn’s change at its end (see below).
  • agent_progress {topic, progress?}: a chat’s Progress changed.
  • messages {messages}: a conversation’s transcript history, added at once.
  • buttons_cleared {topic,id}, outcome {topic,id,outcome,reason?} (accepted/rejected; a refused message’s reason follows in a second outcome), delivered {id} (the router consumed owner input), read {topic,upto}.

A message has id, topic (absent for the root chat), from (agent/owner), ts (ms), kind (text, image, video, audio, speech, voice, document, album), text, and format (plain, markdown, question, progress, progress_card, progress_update, styled). Optional fields: bold (UTF-16 ranges), buttons (rows of {text,data,role?}, where role is destructive or cancel), reply_prompt, reply_to, blobs [{id,mime,size,width?,height?,file_name?,duration?,duration_ms?,frame_rate?}] (duration a voice note’s or spoken reply’s seconds, duration_ms a video’s or audio file’s, frame_rate a video’s), client_id, outcome, reason, receipt, edited, transcript (an owner voice note’s confirmed transcription, added by an edit), interrupted (the owner interrupted the turn from an app after this message; see interrupt), links (web links in the text), same, content keys: a history copy sharing one with a live message is the same message, alert, page, timer, anchor, refs, origin, marks, marked, and spoken (see Marks), page_marks (see Page marks), progress, progress_update, and progress_item (see Progress), and rev. alert (ping, chime, bell, or alarm) marks an agent’s alert: clients play that sound when a live message or edit event first brings it, no more than a minute after the alert was due (timer.ends_ms, else ts), even with notifications muted, and never for a snapshot or history copy. receipt, on an accepted owner prompt, is waiting until the agent has it in its input, then read, or dropped when it never will (Claude refused it after Escape in its terminal, or the agent exited first): clients mark a dropped one not delivered and offer to send its text again, as a new message. Each change comes as an edit; one without receipt is untracked or no longer known. Photos, albums, and slash commands carry none, nor do history copies. reason, on a rejected owner message, says why the worker refused it; clients show it from the message’s failure mark. The chat gets no notice for it, nor for a change that only reports the agent’s state (resumed, unknown, a question only its terminal can answer), which the topic state shows. page is the title of the page an owner message was sent from. timer {id,ends_ms,state} is an agent’s countdown: clients count down to ends_ms while state is running; done and cancelled end it. anchor, on the first message of each agent final reply, live and history copies alike, is what a reference link names that reply by (see References). refs [{link,topic?,anchor?}] lists the vendetta://conversation/… links in the text, in order and without repeats: link as written, topic the app chat of that conversation when there is one, and anchor the reply it names. origin {kind,name?,topic?}, on an owner-side message (from: "owner") the owner did not send, live and history copies alike, says who sent it in their place: kind agent (another agent’s vendetta agent send), worker (Vendetta itself: a manager’s brief and its team’s updates), plugin (a plugin’s plugin_prompt, name its label, such as “Changes”), schedule, or dictation, or person (another person a project share lets prompt, name their display name, via {kind,name?} what of theirs sent it: app, terminal, agent with its title, dictation, or schedule); name what to call the sender (“Team update”, the sending agent’s title, the schedule’s ID); topic the sending agent’s chat when it has one. In a shared chat, history copies of prompts carry origin person too, read back from the worker’s sender line, which the text leaves out. Such a message is markdown with links and refs as an agent reply has. Clients show it as a compact card labeled with its sender, not the owner’s bubble: it has no delivery marks, receipt, or Resend, and is not among the owner’s own messages. Clients show any worker one as Markdown, and a team update (text starting Vendetta team update:) one line per member: its title linking to its chat, the event, the reply’s link, and the quote, without the preamble, provider, project, or conversation key the manager’s copy carries. Without origin, the owner sent it, or no one knows. Clients derive what agent text reads aloud when they read it, with the shared kit (crates/kit), as the server’s narration does; fixtures/kit/speakable.json holds cases.

The server prepares what clients show:

  • A running turn (progress) carries each separate agent response in responses, shown as its own message, and nothing else: the agent’s todo list is its Progress. thinking lists the indices of responses that are the agent’s readable thinking, plain text that clients style apart or hide by a device setting. The app always receives thinking; Telegram and Discord follow their progress.reasoning. A turn whose progress outgrows a message’s 8,000 UTF-16 units continues in a new progress message below; the full one keeps every response it showed. A Progress card or update posted during the turn ends its message the same way. Final replies are markdown; nothing else is.
  • rev is a message’s revision: 0 (absent) when sent, greater with each change, and greater across server restarts. A change that keeps a running turn’s responses up to some index comes as a progress event instead of an edit: on a held copy whose rev is base, keep responses and same up to from, append the event’s (one key per response), take its thinking (all indices), its links and refs (all, absent meaning none), then set rev and edited. An event whose rev the held copy already has (not above it) changes nothing. A client holding the message at another revision, or not at all, changes nothing, sends get_message {topic?, id}, and holds later progress events for that message until message_copy {topic?, id, message?} answers: the message as it is now, with no seq, or no message when the server no longer has it. A copy not older than the held one replaces it; then the held events whose base follows apply in order, and the rest are dropped. The batch after a welcome never carries progress events: each message one changed comes whole as an edit. A later batch carries a message’s progress event only when it is that message’s one event there.
  • A question menu carries question: step/steps (absent on the review page), review, prompt, multi, freeform, options [{number,label,detail?,recommended,selected,data}], actions (custom, note, done, back, submit, and cancel buttons), extras (the custom answer and note given so far), and submits_on_pick. Picking presses the option’s data. Messages live in the plugin’s memory, so after it restarts a question still open comes again as a new menu. A menu that takes no more answers (answered, cancelled, or resolved in the terminal) loses its buttons and keeps its card with resolved: true; a buttons_cleared event says so for a menu a client holds.
  • The receipt for an answer the agent took carries answered: cancelled and items [{header?, question, choices?, custom?, note?}], each question as asked with what was picked or typed, beside its plain text.

When the plugin first sees a conversation, it adds that conversation’s recent prompts, in-between responses, and final replies from the agent transcript as one messages event of messages with history: true; an in-between response or thinking block is a progress message with that one response, thinking: [0] for thinking. A prompt the chat did not send, as from the Master, the agent’s terminal, or a schedule, joins the same way once the plugin sees its turn running or its reply. Their IDs follow their transcript time and stay the same across restarts. They never count as unread and never trigger a push. Each message carries same, keys of its shown content (a voice note’s transcript, each response of a running turn); a history message that shares a key with a live message is that message, so clients drop it, and a live message replaces history copies it shares a key with. fixtures/app/ holds generated frame sequences with the state clients must reach, and for some, progress (the rev and responses chosen messages end with) and requests (the get_message requests the client sends).

An agent’s todo list, from its harness’s todo tool or vendetta agent plan, is its Progress. The server compares each list the agent shows with the last one its chat showed for that conversation and process: items match by text (trimmed, inner whitespace collapsed, case folded), each new item taking the first unmatched old one, whatever its phase or place.

Comparison When Strip Chat
Created No list before (none yet, cleared, or a new conversation or process), one now The new plan The card
Replaced Fewer than half of the larger list’s items match The new plan, new id The card
Changed An item added, removed, or of another status Updated An update
Same Reordered, regrouped, renamed phases, links changed Updated Nothing
Cleared A list before, none now Gone Nothing

A list already there when the plugin starts shows on the strip without a card.

Progress {id, rev, summary, phases:[{name, items}]}: id is new when a plan is created or replaced; rev grows with each change the strip shows, across restarts. An item is {id, text, status, link?, change?}: id (1, 2, … within its plan, never reused) stays while the item continues; status pending, active, done, blocked, or abandoned; link who does it, a reference as refs lists one ({link, topic?, anchor?}); change the change holding its work, vendetta://page/diff/change-ID. summary {done, total, active, pending, blocked, abandoned, now?, next?, last_done?}: total counts every item but abandoned ones, now is the first item in progress, next the first pending one after now (else the first pending one), last_done the last done one, in list order.

  • The strip. snapshot.progress and agent_progress carry {topic, progress?}, progress absent when the plan was cleared or its conversation changed. A client keeps the newest per chat: one whose progress.id is the held one’s and whose rev is not above it changes nothing. Clients show the strip above the composer while the chat has a progress, whatever the agent’s state: collapsed done/total, now (else next, else last_done), and blocked when not 0; opened, every item with its status, link, change, and a reply button. The strip shows no tool call.

  • The card. An agent message, kind: "text", format: "progress_card", with progress: the plan as it was when created or replaced, never changed after. text is the plain list: Progress 0/7, then one mark text line per item. Each item has the strip’s reply button, but for one no longer in the current plan (progress.id changed, or its id gone).

  • An update. format: "progress_update" with progress_update {plan, changes: [{kind, item}], done, total}: kind added, removed, pending, started, done, blocked, or abandoned; item as it is now, as it was for removed; in list order, status changes and additions, then removals. text is the plain line: Progress 3/7 · ✓ Fix login · ▶ Review · + Docs · − Old step. While the update is the chat’s newest message, the next change edits it (edit, a new rev), its changes counted from the list before it; once anything else posts, the next change posts a new one.

  • Cards and updates notify no one and count as read. Marks: ○ pending, ▶ started, ✓ done, ⚠ blocked, ⊘ abandoned, + added, − removed. Clients render them and derive nothing.

  • Replying to an item. send takes progress_item: {plan, item}, the plan’s id and the item’s; with reply_to the send fails. The server resolves it against the chat’s current plan, failing a stale one with send_failed “That Progress item is no longer in the plan”. The owner’s message carries progress_item {plan, item, text}, the item’s text then, which clients show as the replied-to quote. The agent’s prompt quotes the item as a JSON string, with its status then and, when the plan has several phases, its phase:

    Replied-to item of your Progress, now done, under "Release" (quoted context):
    "Fix login"
    
    Current message:
    Did you also cover the expired-token case?

The overview’s todo {done, total} counts as summary does. fixtures/app/progress-plan.json holds the frames the server sends for one chat, with each step’s strip and an item reply’s prompt.

Request Reply
send {client_id, topic, text?, reply_to?, progress_item?, mentions?, attachments?:[{kind,mime,data?,upload?,file_name?,duration?,marks?}], marks?, page?} sent {client_id,id} or send_failed {client_id,message}
upload_start {upload, kind, mime, size, file_name?} upload_status (see Uploads)
upload_chunk {upload, offset, data} upload_status
upload_cancel {upload} none
press {topic, message, data} later toast {text} and router output
read {topic, upto}, or read {reads:[{topic, upto}]} for up to 2000 chats at once (a project’s or a team’s Mark as Read), saved once a read event per chat that moved
fetch {blob, request?} blob {blob,mime,data,…}; a refusal is error with that request as id; videos use media
media {request, blob} media_result {request, ok, url?, expires_ms?, gone?, message?} (see Photos, videos, and audio)
presence {foreground, topic?} none; foreground devices suppress pushes
push_token {token, sandbox, bundle?} push_registered; bundle is the app’s bundle identifier, the APNs topic its pushes use
live {start?, activities: [{key, token?}], status?, info?} live_registered {start, activities, status}; with features.live_activities (see Live Activities)
web_push_subscribe {key, subscription} web_push_registered (see Web Push)
web_push_unsubscribe web_push_unregistered
tasks tasks {available, tasks, timezone, as_of, pending} (while a plugin serves tasks)
task {request, action:"mutate", args} later task_result
schedule {request, action, args?} later schedule_result
queue {request, topic, action, id?, text?, index?} queue_result {request, ok, id?} and a queue event
narrate {request, text} narration_part {request, index, mime, data, length} per piece, then narration_result {request, ok, parts?, message?}
agents {request, action, args?} later agents_result
page {request, action, args} page_result {request, ok, data?, message?}
timer {request, action:"cancel", id} timer_result {request, ok, message?}, then an edit
get_message {topic?, id} message_copy {topic?, id, message?}
older {request, topic?, before, limit?} older_result {request, topic?, before, messages, more} (Synchronization)
ping pong

The server sends a WebSocket ping every 20 seconds and closes a connection that sends nothing for 70 seconds; a pong to its ping counts, so a client with nothing to say stays connected.

mentions lists the agent chats ({kind:"agent", id}) and hub tasks ({kind:"task", id}) the text names as @Name. The server appends a “Vendetta references:” block telling the agent how to find each one the text still names; a queued message takes mentions the same way. A send also names its own photos: {kind:"photo", id:"N"} is @Photo N, its Nth attachment from 1, which must be a photo; the block calls it album member N when there are several attachments, as the agent receives them. The message keeps the block; clients show its text without it, each @Name it lists a link: an agent’s to its chat as the reference to its conversation_key, a task’s to that task, a photo’s (vendetta://photo/N) to the message’s own Nth attachment.

Attachments are base64. Photos must be JPEG, PNG, or WebP up to 8 MiB; voice notes are AAC .m4a, WAV, MP3, Ogg Opus, or WebM Opus up to 32 MiB and must be sent alone; documents are up to 20 MiB inline, or 256 MiB uploaded first, as videos always are. The server measures each recording, and its length is what the message’s blob duration shows, whatever the client declared. A voice note is never refused for its length: past features.voice_max_seconds only its first part is transcribed and sent, and a notice in the chat says how much. A voice note that fails before it reached the agent is rejected with a reason saying why; clients keep its recording until it is accepted and offer to send it again. Several attachments form one album, which goes at once: the server has all of it. A press is accepted only for a button currently on that exact message. reply_to names a message in the same topic, or one response of a running turn as {id}~{index}. A reply to a question prompt answers it, one to media sends that media along, and one to any other text gives the agent up to 2,000 characters of that text as quoted context. A reply_to the server no longer holds fails the send, except with marks.

A video the owner sends, and a document too large to send inline, is uploaded in pieces first, then named by a send or agents create attachment {kind, mime, upload, file_name?} (kind video or document), which carries no data. Either is up to 256 MiB. A video must be what an agent’s video may be: video/mp4 or video/quicktime, H.264 or HEVC with AAC audio or none; clients convert anything else before uploading it, or send it as a document.

  • upload_start {upload, kind, mime, size, file_name?} starts or resumes one. upload is the client’s ID for it (1 to 64 characters, unique on that device), size its bytes. The same ID with another kind, mime, or size starts over.
  • upload_chunk {upload, offset, data} adds data (base64, at most 4 MiB decoded) at offset, which must be what the server holds so far; any other offset adds nothing.
  • Both reply upload_status {upload, received, size, chunk, complete?, message?}: received the bytes held, from which the client continues, and chunk the piece size to send. Once received reaches size the server checks it (a video as above) and replies complete: true, or message saying why it was refused. A message (an unknown upload’s chunk, a refused or evicted upload) means the server no longer holds it: the client starts it again from 0 or gives up and says why.
  • upload_cancel {upload} removes it. Clients cancel an upload when the owner removes the file, once the message that named it is accepted, and once an agents create naming it replied.

A send naming an upload that is not complete, or of another kind, fails. A complete upload stays until cancelled, so a failed send is sent again without uploading again. Uploads are held per device, privately, at most 4 on a device (starting a fifth removes its least recently used) and 1 GiB in all; one untouched for 24 hours is removed, and none survive a plugin restart. A connection that drops mid-upload resumes with upload_start. A send’s uploads are at most 1 GiB together.

An uploaded file is kept like an agent’s media, a private copy named by its hash, its blob an m- ID that media streams. A video shows as a video message (an album with its other attachments), played and marked like an agent’s. A document shows as a document message; its URL always answers as a download (application/octet-stream, Content-Disposition: attachment). The agent gets each as a private local file the prompt names with its quoted metadata, kept 24 hours, as inline documents; a project machine’s agent cannot take one yet.

A Claude Code session’s main-loop permission dialog comes as a message naming the call in its text, Approval needed, with Allow once and Deny buttons bound to that dialog of that process, and approval: tool, and when known title (what the call is for), code (the command or other argument), path, old and new (an edit’s text and its replacement, or a write’s content as new), and reason (why the provider asks). A press answers it once (answer_agent_approval): its buttons go and its approval gains decision, allowed or denied. Allowed, the call ends unrun and the agent makes it again, which runs once; denied, it is refused. When the dialog closes in the terminal or changes, the buttons go with decision: closed; a buttons_cleared event for an approval a client holds means the same. Other dialogs appear only as topic state and a notice.

An agent’s final reply that links local images (as before) or videos, .mp4, .m4v, or .mov (absolute, file://, or relative to its working directory; at most 10 of each), brings each as an image or video message after the reply’s text. A video’s one blob is {id,mime,size,width,height,duration_ms,frame_rate?,file_name}: video/mp4 or video/quicktime, H.264 or HEVC with AAC audio, up to 256 MiB (1 GiB for a reply’s videos together), width and height as displayed (rotation applied). frame_rate is its first video track’s frames a second, its samples over their length (stts in the track’s timescale) to three decimals: exact for a constant rate (29.97 for 30000/1001), a variable rate’s average, as AVFoundation’s nominalFrameRate; absent when the track lists no samples (a fragmented file) or the rate falls outside 1–1000, where a frame step assumes 30. What becomes of each is decided once: a later attempt to deliver the reply shows the same, a copy removed since listed with size: 0. Other videos, and a machine agent’s, stay links, with a line beside the reply saying why. An album may mix images and videos; clients tell them apart by mime. Clients make poster frames themselves, and the gallery lists videos with images.

Audio files linked the same way, .m4a, .mp3, .wav, or .flac (and an .mp4, .m4v, or .mov without a video track; at most 20 per reply), each come as an audio message in link order with the reply’s videos. Its one blob is {id,mime,size,duration_ms,file_name}: audio/mp4 (AAC), audio/mpeg, audio/wav (8-, 16-, or 24-bit PCM, or 32-bit float), or audio/flac, up to 256 MiB, a reply’s videos and audio 1 GiB together; ALAC, Opus, and other formats stay links with a line saying why. file_name is the linked file’s name, and the message’s text the link’s text when that says more ([soft](…/whoosh-1.wav) gives soft), else empty. Only a reply’s first audio message makes a push (“Audio whoosh-1.wav”); all count as unread. Audio is never in an album or the gallery. A speech message is something else: the agent’s reply read aloud, an in-memory blob fetched with fetch, duration in seconds.

.srt and .vtt links in the same reply are a video’s tracks: each goes with the video whose name it starts with (intro.narration.srt with intro.mp4), or the reply’s only video, and is listed on that video’s blob as subtitles, [{id,file_name,role,label,language?,cues}]. role is narration (the voice-over’s words), onscreen (text drawn on the frames), or subtitles, from words in the name (narration, vo, onscreen, titles, captions…); language a two-letter part (bg, pt-BR); label the link’s text when it says more than the name, else the name’s other words, else the role. At most 6 tracks a video and 16 links a reply; UTF-8 SRT or WebVTT up to 1 MiB, 5,000 cues in time order, each 1,000 characters and 8 lines, ending at most 5 s past the video. One that does not fit stays a link with a line saying why. Each is kept as a small WebVTT (WEBVTT, then per cue its number, times, and plain text, escaped) that media grants and the URL serves as text/vtt; charset=utf-8; cues is 0 once the copy is gone.

The server keeps every photo the app shows, the agent’s and the owner’s, every video and audio file, and the owner’s voice notes and spoken marks’ clips, as a private copy named by its SHA-256 (2 GiB and 512 files in all, the oldest removed first). Their blob IDs, m- and 32 hex digits, survive restarts, and always show what the message first showed, whatever happens to the file later; the owner’s voice recordings have v- IDs instead, so they stay voice notes, never an agent’s audio file. A removed copy is gone, never replaced by another. fetch serves m- images and v- recordings whole; videos and audio are streamed.

A history copy of a final reply carries its photos, videos, and audio as blobs on that Markdown message: the copies its live delivery kept, or, for a reply never delivered live, its files as they are when first seen. A removed copy is listed with size: 0; clients show it unavailable. Each live image, video, and audio message of a reply has one content key in same, r and 16 hex digits, which its reply’s history copy also carries: a client holding the reply’s live messages drops the history copy, as with any shared key, so no photo or video shows twice or goes missing.

media {request, blob} takes an m- blob and replies media_result with ok: true, url, a path to resolve against the endpoint’s origin (wss:// as https://), and expires_ms, an hour away; or ok: false with message, and gone: true only for a copy kept once and removed since. The URL is for this device and blob only; revoking the device ends it at once, a response being sent included. Ask again whenever a player opens and within 5 minutes of expiry. It needs no headers or cookies, so AVURLAsset, <video src>, and <audio src> use it as is. GET and HEAD answer 200 (also for several ranges at once), 206 for one byte range (the whole range asked for, open-ended ones included), or 416, with Accept-Ranges: bytes, the content type, and an ETag; 404 means the URL is unknown, expired, or revoked (ask again), 410 that the copy is gone. The same URL with ?download (any value) answers alike with Content-Disposition: attachment, named as the blob’s file_name (a track’s with .vtt), so a browser’s Save keeps the file under its name; without it the file is served inline. Media connections keep alive (15 s idle, 100 requests, no new request after 10 minutes), up to 32 at once beside the app’s own; past that a connection gets 503 with Retry-After. A response goes on at its reader’s pace for as long as its URL lasts, waiting up to 5 minutes for a reader that takes nothing, as a paused player does.

A send replying to a message with photos, videos, or audio may carry marks, the owner’s notes on places in them, numbered in the order shown across the message, and the pictures that show where:

{"type":"send","client_id":"c-81","topic":"42","text":"Fix these.","reply_to":"1793959200123",
 "marks":[
  {"number":1,"blob":"m-3f2a…","note":"Logo pops in too early","at_ms":3200,
   "rect":{"x":0.1,"y":0.2,"w":0.32,"h":0.35}},
  {"number":2,"blob":"m-3f2a…","note":"Make this part slower","start_ms":5000,"end_ms":7500},
  {"number":3,"blob":"m-9b1e…","note":"Label overlaps the axis",
   "stroke":{"width":0.02,"points":[[0.30,0.10],[0.32,0.12],[0.48,0.22]]}}],
 "attachments":[
  {"kind":"photo","mime":"image/jpeg","data":"…","marks":[1]},
  {"kind":"photo","mime":"image/jpeg","data":"…","marks":[3]}]}
  • 1 to 20 marks, number 1 to N in order, note up to 1,000 characters (newlines allowed, may be empty).
  • Each blob is an m- photo, video, or audio file this chat showed, live or in a history copy. When the server still holds the reply_to message, at least one is on it (a reply’s photos and videos are messages of their own, so one send marks them all); after a restart, a reply_to it no longer holds is fine.
  • On a photo: rect {x,y,w,h} or a highlight stroke {width,points} (2 to 64 points, width up to 0.25 of the image’s width), 0 to 1 of the image as displayed, from its top left.
  • On a video: at_ms (a frame) or start_ms and end_ms (a segment), within duration_ms, and optionally a rect or stroke on that frame or the segment’s first.
  • On an audio file: at_ms or start_ms and end_ms, within duration_ms; never a rect, stroke, or picture.
  • On a cue of a video’s track (track, its m- ID, and cue, its number from 1), never a region or picture: cue_edit change with suggest, the new text (1 to 1,000 characters, 8 lines, not the cue’s own); remove; add with suggest and at_ms, where it starts (from the start of cue cue, the one it follows, 0 for before the first; no end); or none, a note on the cue, which then has a typed or spoken note. All but add carry the cue’s own start_ms and end_ms. One mark per cue, besides new cues. A send’s typed block stays under 48 KiB. marked lists the video with the tracks its marks name.
  • attachments are only the marked pictures: JPEG or PNG, up to 2048 pixels long, one per marked photo or video frame, showing its highlights and their numbers, never the notes; marks lists the numbers it shows. Every mark with a place is on exactly one picture. At most 10 pictures, 8 MiB each and 16 MiB in all.

Anything invalid fails the whole send with send_failed; nothing reaches the agent. Then it goes as any photo prompt does, with the same guards.

The owner’s message is text with marks, each blob as sent, and marked, those blobs in first-mark order (not in the gallery); clients draw the marks over them. The pictures are not on it.

The agent receives the owner’s text, then a block, then the pictures as images (or private files for agents that take none):

Fix these.

Vendetta marks: the owner marked 2 things you sent. Act on each numbered note. Images 1-2 below show where each mark is, with its number, not its note.

Video 1 intro.mp4
- Sent from: /home/owner/site/out/intro.mp4
- Pinned copy: /home/owner/.local/state/vendetta/notifications/media/3f2a….mp4 (what the owner saw)
- 1920x1080, 0:12.480 (12480 ms)
- Mark 1 at 0:03.200 (3200 ms); rectangle x 0.1000-0.4200, y 0.2000-0.5500 (pixels 192,216 to 806,594); image 1. Note: "Logo pops in too early"
- Mark 2 from 0:05.000 to 0:07.500 (5000-7500 ms). Note: "Make this part slower"

Photo 2 plot.png
- Sent from: /home/owner/site/out/plot.png
- Pinned copy: /home/owner/.local/state/vendetta/notifications/media/9b1e….png (what the owner saw)
- 1600x900
- Mark 3 highlight over x 0.2900-0.4900, y 0.0822-0.2378 (pixels 464,74 to 784,214); image 2. Note: "Label overlaps the axis"
  Highlight path (width 0.0200): 0.3000,0.1000 0.3200,0.1200 0.4800,0.2200

An audio file is an item of its own, named by its file, so its marks say which sample they mean; its copy is (what the owner heard) and its one line its length:

Audio 3 whoosh-2.mp3
- Sent from: /home/owner/film/sfx/whoosh-2.mp3
- Pinned copy: /home/owner/.local/state/vendetta/notifications/media/0e9d….mp3 (what the owner heard)
- 0:01.850 (1850 ms)
- Mark 4 at 0:00.420 (420 ms). Note: "Cut the attack here"
- Mark 5 from 0:00.900 to 0:01.600 (900-1600 ms). Note: "Fade the tail"

The block’s images sentence appears only when pictures follow.

A video with cue marks lists each marked track, then says per mark what to do to which cue, with the cue’s text and the owner’s on lines of their own; the block’s first line says what a change means for each role it touches:

- Track "Narration" (narration) intro.narration.srt
  Sent from: /home/owner/ads/out/intro.narration.srt
  Pinned copy: /home/owner/.local/state/vendetta/notifications/media/5d1c….vtt (normalized WebVTT, what the owner saw)
- Mark 1: change narration cue 3 of "Narration" (intro.narration.srt), from 0:08.200 to 0:10.900 (8200-10900 ms).
  Current:   "Most people who hate Teslas have never even driven one."
  Suggested: "Most of them never drove one."
- Mark 2: remove narration cue 1 of "Narration" (intro.narration.srt), from 0:00.000 to 0:02.500 (0-2500 ms). Note: "too much"
  Current:   "Hey."
- Mark 3: add a narration cue to "Narration" (intro.narration.srt) after cue 3, at 0:11.500 (11500 ms).
  Suggested: "Link in bio."
- Mark 4: note on narration cue 2 of "Narration" (intro.narration.srt), from 0:03.000 to 0:05.400 (3000-5400 ms). Note: "different footage under this line"
  Current:   "Scroll the comments on any Tesla post."

A history copy of the prompt is read back from that block: its text is what precedes it, and it carries the same marks and marked (a copy no longer kept with size: 0). A block that does not read back whole stays plain text. The live message and its history copy share a content key over the block.

A marked send may carry the owner’s voice as voice attachments (the formats of voice notes, with duration in seconds):

  • {"kind":"voice",…,"marks":[2]} is mark 2’s note, spoken: at most one per mark. Its typed note may be empty; with both, the agent gets both.
  • One voice without marks is the reply’s text, spoken, in place of text (which must then be absent).
  • Together the clips run at most features.voice_max_seconds, as the server measures them (5 seconds’ grace), and 16 MiB, beside the 16 MiB of pictures; a longer send fails with send_failed, never cut, so no mark loses its note.
  • Without features.transcription the send fails with send_failed; clients check it before offering to record.

Before the prompt goes, the server transcribes the clips with the configured [plugins.speech.transcription], two at a time across every chat, with the same checks of the audio and its text as a voice note. Other chats are served meanwhile; the same chat’s later messages wait and go after it, in order. A restart meanwhile rejects the send with a notice, and reports the messages waiting behind it as unsent. Each clip keeps the transcriber’s own deadline, and the whole send has 600 s from the first clip’s start. Once every transcript is in, the agent is checked again, as after a voice note. Any failure, a changed agent, or the deadline fails the send whole, with outcome: rejected and a reason naming the clip; nothing reaches the agent. Clients keep the draft, recordings included, until the message is accepted.

The owner’s message carries spoken, one {mark?, blob?, transcript?} per clip (mark absent for the reply’s text): blob the clip, which fetch serves until the plugin restarts, as voice notes; transcript arriving with the edit that follows acceptance, which also adds the message’s content key. Its text is empty when the reply was spoken. Clips are never kept as copies.

In the agent’s prompt a spoken reply’s transcript stands where the text would. The block’s first line then adds The owner's message above was spoken., and with any clip, Spoken notes and messages are transcriptions; read past misheard words. A spoken mark’s line ends with its typed note, if any, then Spoken note: "…":

- Mark 2 from 0:05.000 to 0:07.500 (5000-7500 ms). Spoken note: "slow this down to half speed"
- Mark 3 highlight over …; image 2. Note: "Label overlaps". Spoken note: "and move it left"

A history copy reads spoken back from the block, transcripts without blob; a spoken reply’s copy has empty text and its transcript in spoken.

A send may instead carry page_marks, the owner’s notes on lines of a plugin page (Plugin UI: diffs), such as a diff, never with marks, reply_to’s media, or attachments:

{"type":"send","client_id":"c-90","topic":"42","text":"A few things.",
 "page_marks":[
  {"number":1,"plugin":"diff","page":"change-c7f3a-r3","node":"h:2:0","line":4,"end_line":6,"note":"Read this from the config"},
  {"number":2,"plugin":"diff","page":"change-c7f3a-r3","node":"f:2","note":"Split this file"}]}
  • 1 to 20, number 1 to N in order, note 1 to 1,000 characters.
  • node is a node of that open page with mark; line and end_line (default line) index its lines from 0, end_line at most 50 past line; no line marks the node itself. The page is read from the worker as the send arrives; a node it no longer has fails the send with send_failed (“The page changed; mark it again”).

The agent receives the owner’s text, then:

Vendetta page marks: the owner marked 2 places on pages. Act on each numbered note.

Mark 1 on "change c-7f3a r3 · src/auth.rs", new 42 to new 44
- Page: diff change-c7f3a-r3 "h:2:0" lines 4-6
- Lines:
  + let retry = 3;
    let wait = 100;
  + loop {
- Note: "Read this from the config"

Mark 2 on "change c-7f3a r3 · src/auth.rs"
- Page: diff change-c7f3a-r3 "f:2"
- Note: "Split this file"

The item and node are JSON strings. Lines gives each line as the node’s lines hold it, a tag and its text, cut at 300 characters. A page’s root may name chat_key, a conversation; devices get it as chat, its topic, the chat the marks go to. The owner’s message carries page_marks, each {number, plugin, page, item, where?, note} (where its labels, such as new 42 to new 44), and its history copy reads them back from the block, as for marks.

tasks lists the active tasks of the plugin serving tasks with id, hash, title, details (multiline notes), status, priority, scheduled, deadline, snoozed_until, next_reminder, reminder_times, project_ids, and updated_at. A vault that cannot be read completely replies available: false, never a shorter list. While a device is connected, the server sends a changed list as a tasks event as soon as the plugin serving tasks reports a change, so changes made elsewhere appear without asking.

task mutate takes {id?, expected_hash?, mutation_id, patch}, the tasks service’s task_mutate: an edit names the task’s current hash and conflicts if the task changed; a snooze sets snoozed_until alone and never moves a deadline. project_ids lists Vendetta project names (hub project IDs where the hub has that project); a task without one is personal. details is plain text up to 16 KiB; an empty string clears it. Reusing a mutation_id with the same change returns the first result.

schedule actions map to the scheduler tools of the same meaning: list (scheduler_status, each schedule with next_run and its cron in words as description), history, create, update, enable, disable, run, delete, start (the worker), and preview (a draft’s cron, at, and timezone checked, with its next_run and description, for editors). args are those tools’ arguments: create and update take one of cron, at, or on_demand: true (no time of its own: run or a webhook runs it), and one of new: true, reuse: true (the same agent each run, starting from the newest one its runs started), reuse_from (a conversation key to reuse), or session_id. list shows a reusing schedule’s agent as its record’s conversation, and their reply as its own conversation. Personal hub reminders are neither listed nor reachable. A history run whose conversation has an app chat carries that chat as topic. A schedule with source {kind: "file", folder, id, agent?} comes from a project’s vendetta.toml: clients show it read-only; update and delete refuse it with read_only, while enable, disable, and run work.

agents actions project_files (every project file with its agents’ states, as the core’s project_files, each with its hash and trusted (whether its folder is trusted: offer Trust only when false with no error, Untrust only when true); each schedule’s cwd absolute; each agent with its provider, model, effort, manages, and declared manager’s id; each schedule with its timing and target but no prompt; each pending item with its agent or schedule: what the dialogs list), trust_project {folder, trusted, hash}, approve_project_item {folder, kind, id, hash}, both refused with changed when the file differs from the hash the device showed, and when it does not check, (an agent or schedule a trusted file added, which waits until approved), and start_declared {folder, id} are the owner’s; Telegram and Discord offer none of them. While a device is connected, the server sends a project_files {files} event, the same list, whenever the core reports a file found, edited, or trusted, or a declared agent’s state changing, so trust prompts and declared states update without asking; a batch keeps only the last one.

task_result and schedule_result carry request, action, and either ok: true with value, or ok: false with message and uncertain. An uncertain result means the request reached the core but its outcome is unknown; neither the server nor the app resends it. A request refused before it runs replies error with code: "request" and its id. Requests run a few at a time across every device, the rest waiting their turn in order; only when too many already wait (128 in all, 32 from one connection) is one refused with code: "busy", having run nothing: clients ask a read again after a pause, and leave a change for the person to retry.

A request the worker refuses for a project machine (see Machines) carries reason beside message, on send_failed and on any *_result or page_result with ok: false: machine_stopped (start the machine, or resume the conversation, which starts it), machine_unreachable, machine_unavailable, or machine_unsupported (what machines do not do, such as the Master or terminal placement). Nothing reached the machine, so it is never uncertain. message is the sentence to show. A prompt sent in a chat that the worker refuses this way is rejected, with a reason saying why.

Sharing adds these reasons: share_none, sharer_only (only the sharer stops, deletes, resumes a declared agent, or runs that command), share_revoked, machine_busy, peer_unreachable (the sharer’s Vendetta cannot be reached), peer_unsupported, not_container, not_git (offer “Make it a git repository”, which sends the same request with make_git: true), and admin_only.

people {request, action, args?} replies people_result {request, action, ok, value?, message?, reason?, uncertain?} as schedule does. list (every device) replies {people: [{name, display_name, admin, me, state}]}, state linked or unreachable: whom a share can name; a people {people} event follows each change. The rest are the admin’s devices’ only (admin_only elsewhere), the worker’s instance methods:

  • instances → {instances: [{name, display_name, port, web_url, state, worker_running, endpoint_up, current, link, projects: [{name, machine?, old_image?}], address?}]}: each person’s projects and machines, never their chats. address, while the Tailscale plugin runs: {state: "answering"|"waiting"|"unavailable"|"off", service?, web_url?, port_web_url?, service_web_url?, note, policy?}; show note as is, say “Waiting for approval” while waiting, and policy as code.
  • add {name, display_name?} → {job}; then jobs events until it is done, which waits for their Vendetta to link and carries its first pairing (as pair gives it): show it at once. A job done without one (their Vendetta was slow to start) offers pair.
  • pair {name} → {code, qr, web_url, expires_ms}: show the code and a QR of qr to hand over, with the web address and that their phone needs tailnet access.
  • update {name?} → {job}.
  • What a person takes from the admin (Claude, Git, Services, their Master, each a switch) and their own connections’ state come from the worker’s people contribution, drawn in their card; their skills and MCP servers from the Claude proxy’s, a row opening its page. For each person, and “Your container projects” (_machines), open PREFIX-NAME (the people slot).
  • remove_plan {name} → {plan, lines}; show the lines (their folder goes: their conversations, chats, paired devices, and container projects), have the name typed, then remove {plan} → {job}.
  • rename {name, display_name}.

A paired admin device can add and remove people and their containers: revoke a lost one at once.

projects {request, action, args?} replies projects_result the same way; args takes instance for another person’s (admin only):

  • list → {projects: [{name, state, repo, origin, image_current, memory, agents, unfetched?, shares?, error?}]}, state running, stopped, unreachable, unavailable, or absent; origin existing, new, or clone.
  • create {from, make_git?}: from {existing: folder} (“Run in a container”), {new: name}, or {clone: url, name?} → {job}. A name goes as typed; the worker folds it into the machine’s name (fixtures/machine-names.json), and clients preview that under the field (“Makes ~/projects/test626”), with an error only when nothing is left.
  • start, stop, recreate, fetch, update {name, ref?} as their machine methods.
  • finish {name} → {job}: a project absent with no create job running was never finished (its create failed, or a restart cut it off; the worker finishes the latter itself as it starts). Offer Finish creating, which this asks, and Remove; its machine actions are refused until then.

A machines {} event says a project machine changed state (started, stopped, recreated, linked again): read the container projects shown again.

  • remove_plan {name, delete_folder?} → {plan, lines, warnings, folder, folder_optional, delete_folder}, then remove {plan} → {job}. The container, its volumes, and its clone always go, and its folder when Vendetta made it; for a folder the person brought (folder_optional) offer “Also delete FOLDER”, off by default, which plans again with delete_folder: true. Show warnings (also among the lines) plainly before Remove: work that would be lost (“3 changes and 2 commits in FOLDER aren’t pushed anywhere and will be lost”). The lines name each of the project’s conversations, one that runs as stopping; once the machine is gone they are deleted as a confirmed deletion, their chats with topic_deleted.

jobs {jobs} events carry every job’s progress, [{id, kind, instance?, target, step?, steps, labels, step_label, state, error?, started_ms, ended_ms?, title?, pairing?}]: show title (“Turning Claude on for Ana”) in place of the kind’s words when present, each step by its labels entry (the words for that step in this kind of job) and the current one by step_label (“Starting” before the first), the failing one’s error, and offer Retry, the same request again. A job with no instance acts on the admin’s own machine (adding a person is one); one with no step is starting. Events are not replayed: a device asks jobs {request, action: "list"} when it connects, as it does people and shares, and jobs_result’s value is {jobs}.

shares {request, action, args?} replies shares_result the same way:

  • list → {given: [{id, project, person, name, created_ms}], received: [{id, project, group?, by, person, state}]}. Each received project is a group here (group, its shared_by the sharer) whether or not any agent ran in it yet: list it, and offer it in New agent while state is linked; an agent started with its group starts at the sharer’s. person is the sharer’s person name, which canvas_color takes to colour the project.
  • share {project, person}: a whole container project, in full (not_container otherwise; clients offer Share only on those). The share sheet is per project: pick a person, Share, and the current shares with Remove.
  • unshare {id}: the sharer’s.
  • leave {id}: the receiver’s.

There is one kind of share, a project, and one level of access, so shares carry neither a kind nor a role.

A shares event with the same lists follows every change. A share that ends removes its project’s chats and group with topic_deleted events on the receiver’s devices; their own history copies stay with them. A page a shared reply links is copied here as PERSON--MACHINE--PAGE, its state this person’s own; its images come inline, kept in this person’s media store. A shared chat alerts this person’s devices as their own chats do, with their own mutes.

In a shared chat (shared or shares on its topic), @ lists the chat’s other people first, then what it lists elsewhere. A message that names any of them with @Name (the kit’s people_named, display names, longest first) is a note: it goes to the people of the chat, never the agent, and the composer says so while it does (“Only people see this; the agent won’t.”). A note carries no attachments, marks, or reply.

notes {request, action, args} replies notes_result the same way as shares:

  • people {topic} → {topic, people: [{name, display_name}]}: who else sees the chat, this person left out. not_shared for a chat no one else sees.
  • add {topic, text, mentions: [name]} → {topic}: the note is written at the chat’s sharer and arrives as a message like any other. not_in_chat when a mentioned person no longer sees it.

A note is a message from owner with note {by, mentions}, each {name, display_name, me?}, me on this person. It has no delivery, receipt, or Resend, and is not among the owner’s own messages; clients draw it as a card between people, headed by the kit’s note_heading (“Ana to you”). Notes written before the plugin started come as history, and a note keeps one content key (note:ID). A note that mentions this person, by someone else, pushes to their devices with the chat’s title and “NAME: text” (“NAME mentioned you” without previews), as their own chats do: never while a device shows the chat, never in a muted chat. Telegram and Discord post a note written while they run as one plain message.

What a person connects is plugin UI (Plugin UI, slots): Settings → Connections (Simple and Advanced) draws each connections contribution’s page inline, own ones under “Yours” and the admin’s under “From the admin” (by the origin of its provider node), “Nothing to connect here yet.” without any; New project offers each new_project contribution as a source. No client names a provider.

The provider node: {name, description?, icon: {symbol?|image?}, origin: "own"|"admin", state: {status, account?, detail?, flow?}, actions: [{id, on}], items?}.

  • icon.symbol is one of code, git, cloud, key, mic, sparkles, server, person, crown, box (draw any other as box); icon.image a PNG or SVG data URL (draw anything else as box).
  • status: not_connected, connecting, connected (with account), error, shared (given by the admin), off, unavailable. Show an unknown one as its detail.
  • flow, while connecting: device_code {code, url, expires_ms} (show the code with Copy and open url), open_url {url, expires_ms} (open it), secret {label, placeholder?, help_url?} (a password field, sent by the submit action with context: {secret}; the secret is passed to the plugin and kept nowhere else). Each has Cancel. Open only https:// addresses, or http:// on loopback. An unknown kind says “Update the app to connect this”.
  • actions: each id the client knows (connect, disconnect asking first, cancel, submit) is a button with the client’s own words, ui_action with item its index; skip the rest.
  • items, when present: read-only rows {id, title, subtitle?, status, detail?}, on their own screen, grouped by subtitle with a search box once there are more than 8.

A new_project source lists with ui_action (no page) and context: {query?, cursor?}, answered with items: [{id, title, subtitle?, private?, updated_ms?, clone_url}] and cursor?; the picked item is made with projects create {from: {clone: clone_url}}.

Buttons and pages plugins add (Plugin UI has the slots, node kinds, and how to draw them). Nothing here is replayed after a reconnect: open pages again.

  • ui {version, contributions} follows each snapshot or welcome, and comes again on change: each contribution {plugin, id, slot, label, icon?, badge?, home?, modes?, order?, when?, on}.
  • ui_open {request, plugin, page} replies ui_page {request, plugin, page, rev, title, root, nodes}, or {request, plugin, page, gone: true}; then ui_patch {plugin, page, rev, set, root?, title?} while it is open, each rev one past the last (set maps a key to its new node, or null when it goes). Anything else is a gap: open it again. A page that goes sends ui_page {plugin, page, gone: true}. ui_close {plugin, page} ends its patches.
  • agent nodes carry an OverviewAgent, its topic and todo filled in here.
  • ui_action {request, plugin, page?, node, item?, choice?, context?} replies ui_action_result {request, ok, notice?, on?, items?, cursor?, message?}. context is {topic} from a chat’s menu, a chat of this person’s (the plugin gets its conversation too), {secret} from a secret flow, {query?, cursor?} from a source’s search; nothing else reaches the plugin. choice is the picked choice of the node’s (or with item, that action’s) ask. Show notice, follow on (a page or a navigate).

claude_login {request} (admin devices; admin_only elsewhere) replies claude_login_result {request, ok, value?}, value the worker’s claude_login (protocol): {plugin: "running"|"unavailable", source: {present, expires_ms, refreshed_ms, refresh: "ok"|"running"|"failed"|null} | null, targets: [{id, kind: "machine"|"instance", instance, machine, state, last_used_ms, requests, input_tokens, output_tokens, error}]}, state waiting, ready, active, refused, or error. Status only; it never carries a token. Settings shows the login’s expiry, a failed refresh as “run /login in your Claude”, and one row per target. A claude_login {} event says it changed: ask again.

Each agent chat can hold up to 50 messages for after the agent’s turn. queue actions are add {text} (replies with the new id), edit {id, text}, remove {id}, move {id, index}, resume, and clear. The queue event {topic, items:[{id,text}], sending?, paused} follows every change and is in snapshots under queues. An item’s text keeps its references block; clients show it without the block, and an edit of that shown text sends the edit with the block’s lines for each @Name it still has.

Once the agent has been idle for a few seconds with no other input pending, the first item is sent as an ordinary owner message; the next waits until the agent has run and stopped running again, however its turn ended. A refused message pauses the queue until resume. Queues live in memory only and are lost when the plugin restarts.

welcome and snapshot carry narration: true when a voice service is configured ([plugins.speech.narration]). narrate reads up to 20,000 characters aloud in pieces, so playback starts after the first: a sentence or two, then longer ones. Each piece arrives as soon as it is ready as narration_part with its index from 0 and its own complete recording, base64 data of type mime (audio/mpeg); clients play them in order. Its length is the whole reading’s length in seconds, estimated from the text left to read and the voice’s pace, and all but exact on the last part and on cached replays, so a player shows the full length from the first part. The narration_result after the last says ok: true with the number of parts, or ok: false with a message, possibly after some parts: those still play, but the reading is incomplete. Clients send the reply whole; without a rewrite the server reads only its prose, leaving out code, tables, quotes, and tool lines. Recent texts are cached in memory, so replaying one sends its parts at once without calling the service again. With narration_rewrite: true, a small model first rewrites the text for listening as it streams, so clients send the reply whole, Markdown included.

agents actions map to the agent tools: projects (list_tmux_sessions, replied as sessions:[{name,cwd}] once each by name, most recently used first, as the chooser ranks them), options (launch_options, replied as {default_provider, ephemeral_after_ms, providers:[{provider, models, efforts, team:{model?, effort?}}]}: what create may take), history (list_history, replied as entries:[{id,title,provider,project,state,updated_ms,topic?,conversation_key,kept,ephemeral,deletes_ms?}], live agents first with running ones leading, then by recent activity; an entry with an app chat carries it as topic), create (an optional model and effort from options start it on them; group, a received shared project’s group, starts it there, at its sharer’s; ephemeral: true makes it ephemeral; prompt, its first message; see below), retry {topic} (below), resume (resume_agent with a history id from this server’s latest history), and rename (rename_agent): {topic, name} names the conversation behind a chat, {conversation_key, name} one history lists, and an empty name removes it. The chat’s topic event brings the new title; its reply is {conversation_key, name}.

create checks everything first: a refused create starts nothing and makes no chat. Then it makes the new agent’s chat in the project’s group and replies at once with CreatedAgent {topic}; clients open that chat. The agent starts after the reply, and the chat’s topic carries start until it is ready:

  • phase launching: the worker is starting its process. state is starting.
  • waiting: its process runs and does not take input yet. state is starting.
  • needs_you: a provider dialog in its terminal waits for the owner, such as trusting the folder or a setup question; reason is the sentence to show. state is starting.
  • failed: it did not start; reason says why, and retry: true offers Retry. state is exited.

since_ms is when the phase began. manager: true marks a manager’s start, which goes from launching to ready, never waiting or needs_you. Once the agent takes input and its first message went, start is gone and the chat shows the agent’s own state (running with a first message, idle without, until the agent’s own reports arrive). Clients render start only as the kit’s start_view words it, and show nothing else of the start.

Its first message shows in the chat at once as the owner’s (the caption, its photos and files, or the voice note with its transcript) with receipt: waiting; once the agent takes it, its outcome is accepted and its receipts follow as any send’s. The transcript’s copy shares its same key, so it shows once. If the agent starts and does not take it, the chat is ready and the message dropped, with Resend. Nothing is ever sent twice: a send whose outcome is unknown is dropped too.

While start is present, a send to the chat is refused (reason: "starting"); clients put the composer in queue mode, and the queue goes on after the agent’s first turn. Commands that only read are answered all the same, until the router binds the chat: /read shows the agent’s terminal as its snapshot reply does elsewhere (“Terminal snapshot …”) as soon as the start has a process, waiting and needs_you above all, and says it has no terminal yet while launching; /status says where the start is. Nothing reaches the agent’s input.

retry {topic} runs a failed start again in the same chat and replies RetriedStart {topic}; it is refused unless start is failed with retry. The server picks what runs: a process of the start that still runs is waited on again, else the agent is started again with the same choices. The first message goes with it. Failures that offer Retry: the launch was refused, the process exited before it took input, it took no input within 90 s (waiting) or 30 min (needs_you), the plugin restarted during the start, and a launch whose outcome is unknown, whose reason says to check the terminal first. A plugin restart keeps the start’s choices, never the first message: Retry then starts the agent without it.

Leaving the chat changes nothing: the start goes on. delete {topic} on a starting chat deletes the chat at once and never sends its first message; a process of the start is deleted as the worker’s delete_agent deletes, and one that refuses keeps running and gets a chat of its own. stop and interrupt are refused while start is present. Auto-delete never counts down a chat with a start (no deletes_ms, failed ones too) and never removes it: when an auto-delete pass or an ephemeral end deletes the conversation behind it, the chat stays with its Retry, which starts the agent anew.

create may carry attachments, [{kind, mime, data, file_name?, duration?}] as send takes them (photos, documents, and uploaded videos together, or one voice note; the same formats and limits; no marks), with prompt as their caption. They are checked, documents and videos saved and a voice note transcribed, before anything starts; a refused one starts nothing. The first message reaches the agent through the worker’s guards as a chat’s send would: photos natively where the provider takes images (OMP, OpenCode), else as private local files the prompt names (Codex, Claude); each document and video as a private local file the prompt names with its quoted metadata; a voice note as its transcript, its first features.voice_max_seconds heard. Refused with manager: true (a manager’s first message is its task), a terminal placement, and documents or videos for a project machine’s agent, as send refuses them; nothing starts then.

mute {topic, muted} (mute_agents) mutes or unmutes the agent behind a chat and replies {conversation_key, muted}; every device gets the chat’s topic event. Typing /mute or /unmute in the chat does the same.

keep {topic, kept} or {conversation_key, kept} (keep_agents) keeps a conversation from auto-delete, or no longer, by its chat or, for a history entry without one, its key; it replies {conversation_key, kept}, and every device gets the chat’s topic event. Typing /keep or /unkeep in the chat does the same. A conversation deleted by hand loses its keep.

ephemeral {topic, ephemeral} or {conversation_key, ephemeral} (ephemeral_agents) makes a conversation ephemeral, or no longer, the same way; it replies {conversation_key, ephemeral}, and every device gets the chat’s topic event. Making a conversation ephemeral stops keeping it, and keeping it ends ephemeral.

Teams (see the protocol): create with manager: true, an optional task (the owner’s request), and members (chat IDs) starts a manager with that team, and replies at once with its chat as any create does, its start marked manager. The start ends once the manager was briefed; when it started but was not briefed, the start ends all the same and a notice in its chat says why, the manager and its team standing either way. assign {manager, members} (chat IDs) adds a manager to agents, which keep the managers they had; release {topics, manager?} takes them out of manager’s team, or, without it, out of every team, dismissing managers among them in none; managers {topic, managers} (set_managers) sets a chat’s managers to exactly those chats, [] for none. They reply {manager, members, notified, notice_error?}, {released, dismissed, notified, notice_error?}, and {agent, managers, added, removed, notified, notice_error?} with conversation keys, notified: false with notice_error when a manager could not be told; a link that would make an agent its own manager at any depth is refused, changing nothing. The chats’ topic events bring the new places. create with manager: true also takes team: {PROVIDER: {model?, effort?}}, what the agents it creates run with. team_choices {topic} (team) replies {choices: {PROVIDER: {model?, effort?}}} for the manager behind the chat, and set_team_choice {topic, provider, model?, effort?} (set_team_choice) replaces one provider’s and replies the same; a part left out is that provider’s team in options. info {topic} (agent_info for the conversation behind the chat) replies with what the agent’s own records say about it, each part null when its provider does not record it: model {id,name,provider,effort,detail}, context {used,window} in tokens, session {input,output,cache_read,cache_write,reasoning,cost_usd,responses}, limits [{name,scope,used_percent,window_minutes,resets_at_ms,as_of_ms}], plan, credits, supported, and as_of_ms, when its newest record was written.

commands {topic} (agent_commands for the conversation behind the chat) replies {live, commands:[{name, description, hint, source}]}, sorted by name: the slash commands a message in that chat can run, for autocomplete. name is what follows the /, hint the argument hint or null. The agent’s own come first-hand (live: true) or from its provider’s vendored list, with source builtin, user, project, plugin, skill, mcp, extension, or prompt; names this router handles itself are replaced by its own that work from the app, source: "vendetta": /help, /agents, /status, /progress, /read, /photo <path>, /stop, /delete, /mute, /unmute, /keep, /unkeep, and in an OMP chat /queue <prompt> and /new. Nothing is kept; fetch it again after reconnecting.

stop {topic} ends the live agent behind a chat with the core’s guarded shutdown (close_agent): one SIGTERM to its exact process, refused while a question or approval is open, then a confirmed exit. Nothing is typed into its terminal or added to the chat. It replies {} once the agent exited; the chat’s topic event then shows exited, and a later prompt there resumes the conversation. ok: false means nothing was stopped (no live agent, several, or a question or approval to resolve in the agent’s terminal first), unless it is uncertain. Typing /stop keeps working as before.

interrupt {topic} ends the agent’s current turn, as Escape does in its terminal (interrupt_agent); the agent keeps running and waits for the next prompt. Claude, omp, and OpenCode take it natively, also outside tmux; other agents get Ctrl+C in their terminal. It is refused unless the agent is running, and while a question or approval is open. It replies {} once the agent took it; the chat’s topic event then shows the agent ready, and the chat’s newest message gets interrupted: true in an edit, so every device marks where the turn was cut short. Nothing is added to the chat, and messages sent during the turn and the chat’s queue go on as after any turn. ok: false means nothing was interrupted, unless it is uncertain.

avatar sets a chat’s picture, for any chat but the root: {topic, emoji} an emoji (1 to 32 bytes, no spaces), {topic, image, mime} an image (base64 JPEG, PNG, or WebP of type mime, at most 512 KiB and 1024 pixels a side; clients send a square of at most 512×512), and {topic} alone removes it. It replies {}, and every device gets the chat’s topic event. Pictures stay with the chat across restarts and go when it is deleted.

delete {topic} or {conversation_key} (delete_agent) deletes the agent behind a chat, or a history entry without one, once the owner confirmed it where they chose Delete: it stops a live agent with one SIGTERM once it is idle, waits for its confirmed exit, then deletes the conversation with its app chat, Telegram topics, and Discord channels, with all their messages; its files stay. Nothing is sent into the chat. It replies {conversation_key}; the entry leaves history, the app chat goes with a topic_deleted event, and Telegram and Discord follow once their routers run. It is refused, changing nothing, for the Master, a kept conversation, a manager whose team still has members, an agent working, asking, or awaiting an approval, and another person’s agent; show the message where Delete was chosen. Typing /delete in a chat still asks there; apps never send it. Auto-delete removes a conversation the same way, with the same topic_deleted event.

project_colors (the core’s project_colors) replies {colors: [{project,person?,color}]}, every project colour kept, each live project’s among them (a live project gets one the first time it is asked for), so lists colour projects with no live agent: color 0 to 11, hue 45 + 30 × color degrees at the client’s lightness for its theme. canvas_color {project,color,person?} (person for another person’s shared project) gives a project another of the 12 and replies {}. Clients reread the colours on topic events, after their own colour change, and every minute.

With [plugins.ntfy] app_links = true, an alert’s tap target is vendetta://conversation/HASH, a reference to the chat.

A reference link names an agent’s chat or one final reply in it:

  • vendetta://conversation/HASH, the chat: HASH is the first 32 lowercase hex digits of SHA-256 over the conversation’s conversation_key.
  • vendetta://conversation/HASH/reply/ANCHOR, one final reply: ANCHOR is the first 16 lowercase hex digits of SHA-256 over agent, a newline, and the reply’s trimmed text as vendetta agent messages returns it.
  • <web base>/?conversation=HASH&reply=ANCHOR (reply optional), the same in the web client, <web base> as for pages.

Agents write them as Markdown links, [Login fix is done](vendetta://…), and messages list them in refs. Tapping one, a client opens its topic, or without one the chat of the history entry whose conversation_key hashes to HASH. With an ANCHOR, it scrolls to the message whose anchor matches, the newest if several, among all the chat’s messages it holds, rendering older ones as needed, and highlights it briefly. A reply it cannot find opens the chat at its end, and a chat it cannot find opens nothing; either way a toast says so. Discord gets a link to the chat’s channel, or to the reply’s message when the router delivered it; otherwise Telegram and Discord get the web link when public_url is set, else Discord the chat’s channel, else the link’s label as plain text.

Vendetta installs the vendetta-pages skill, which has agents write small self-contained HTML pages to ~/.local/state/vendetta/pages/ and link them at the end of a reply. A final reply’s links to .html files inside that folder (at most three, up to 2 MiB each, rechecked when sent) arrive as blobs with mime: "text/html" on the message; other transports keep the link. A machine agent’s pages come from its machine and are kept in that folder as NAME--<page>.html, NAME the machine’s; a page it links that the machine cannot give shows as the link’s text alone. A page’s blob ID pins its file’s content, so fetch still finds it after the server restarts or drops it from memory, until the file changes.

A page’s web link is <web base>/?page=<name>: <web base> is [plugins.app] public_url as an origin, wss:// read as https:// (ws:// as http://) and its path dropped, and <name> the file name without .html, URL-encoded; vendetta page link FILE prints it. In a final reply, a web link with exactly that base, written as a Markdown link or bare, attaches its page as a file:// link does, with the same limits. The web client opens such a link with open.

Clients render pages without network access or storage of their own, and give each page this script API:

  • vendetta.send(text) sends text to the chat’s agent as the owner, as send with page set to the page’s blob ID; it works only within five seconds of the owner’s tap or key press on the page, and resolves once the server accepts it. The chat shows the message with the page’s title as its page; the agent gets it after a line naming the page and its file. The viewer shows what was sent and the agent’s reply as it comes. The older webkit.messageHandlers.vendetta.postMessage({reply: text}) does the same.
  • vendetta.state.load() resolves to the page’s saved JSON value, or null; vendetta.state.save(value) replaces it, up to 256 KiB. The server keeps it beside the page as <page>.state.json, so the agent can read it and every device sees the same.

page actions take a page by any of its blob IDs:

  • pin {topic, blob} pins the page to that chat (at most ten; pinning again does nothing) and unpin {topic, page} removes it; both answer with a topic event. Pins are kept across restarts.
  • load {blob} answers with data, the saved value or null; save {blob, data} saves it.
  • open {page} takes a page by its name and answers with data {blob, title, topic}: its current content’s blob ID, its title, and the chat it was last delivered or pinned in, or null for the root chat or none. The server remembers that chat for the newest 500 pages, across restarts.

An agent sounds an alert with vendetta alert TEXT [--in DURATION | --at TIME] [--sound ping|chime|bell|alarm], now or up to a day ahead, or starts a countdown with vendetta timer DURATION [TEXT], which is the same alert set --in DURATION. An alert due now is an agent message with that text and alert set, which counts as unread. One due ten seconds or more ahead is a timer: its chat gets the message at once with timer running, quietly, and unread from then on; when it is due, an edit sets timer.state done and alert. Either way, devices not in the foreground get a time-sensitive push with the alert’s sound when it is due. The owner cancels a running timer with timer {action:"cancel", id} and the agent with vendetta alert --cancel ID; an edit then sets cancelled, and nothing sounds.

With APNs set up, features.live_activities is true and the server shows Live Activities on iPhones through it (apns-push-type: liveactivity, topic the app’s bundle identifier plus .push-type.liveactivity). Both kinds use the attributes type VendettaActivity with {kind: "status"|"agent", key}, key the activity’s id and start (ID@MS); content-state is vendetta_kit::live::Content (crates/kit/src/live.rs): one-word keys, times in integer milliseconds. The device sends live {start?, activities: [{key, token?}], status, info}, tokens in hex, once the snapshot, welcome, or features event says the server takes it, and whenever its push-to-start token or any activity or its update token changes; it replaces the server’s list for the device, which still keeps the server’s starts of the last two minutes the phone has yet to list. status (default true) is the device’s switch for the status activity; info is what iOS says (os, enabled, frequent, push_to_start, start_token, start_token_seen, request, the outcome of the app’s own start), which the server logs with the counts, never a token. The reply live_registered {start, activities, status} says what the server now holds. The status activity is always on for a device whose switch is on: “All quiet” when nothing happens. In the foreground the app starts it itself (Activity.request) when none runs and ends all but the newest; the server push-starts it otherwise. The server push-starts what should show and the device lacks, sends updates to each activity’s token (the status one at most every 20 s at priority 5, at once at priority 10 when something newly needs the owner; an agent’s at most every 5 s at priority 10) with a stale-date 30 minutes out on the status one, ends what should no longer show, and restarts one after 7 h 45 min. Live pushes never sound. A token APNs disowns is dropped. The status activity’s usage is one provider’s weekly and 5-hour limits, or each provider’s weekly, from agent_info, as the apps’ usage strip reads it; a change to it alone is sent only when a provider crosses 20% or 10% left.

What pushes, on APNs and Web Push alike, and what clients notify of themselves: a turn’s final reply, a question, an approval, an alert or a timer that rings, and the pictures, audio, and video an agent sends. A turn’s in-between responses (its progress copy and the responses it carries) never do. A muted chat pushes only its agent’s alerts; the Overview, the status Live Activity, and the home screen’s counts still count its agent.

One rule makes them agree:

  • A notice is titled as the apps title the chat, its project beneath (APNs alert.subtitle), and names its chat and message (topic, message).
  • badge is what the apps count as unread: an agent’s messages past a chat’s read marker, not history copies, in agent chats that are not muted (a timer from its start); never the root chat.
  • A read (from any device, Mark all as read included) and an answered question or approval send every iPhone a clear: APNs {aps: {badge, content-available: 1}, clear} (push type alert, priority 5). clear lists {topic, upto} (that chat’s notices up to that message) or {topic, message} (that one). The badge updates as it lands; the app removes those notices. Browsers must show every push, so they get no clear: an open web page closes them as the read and buttons_cleared events arrive, and a chat’s newer notice replaces its older one.
  • The home screen counts the Overview’s lanes: the lanes {asking, approvals, working} frame comes after a connection’s snapshot or welcome and on each change, counted as the status Live Activity counts them.
  • After a notice, the status Live Activity’s next change skips its 20 s gap, so it agrees with the notice as it lands.

Browsers get the pushes APNs gives the iPhone app: the same notices, at the same moments, worded the same ([plugins.app.web_push] preview, on by default, as APNs’s preview). The plugin makes a VAPID key pair once and keeps it in its state (web-push.json, 0600); features.web_push_key is its public key, base64url, absent when enabled = false. A browser subscribes with that key as its applicationServerKey and sends web_push_subscribe {key, subscription}, subscription as PushSubscription.toJSON() gives it ({endpoint, keys: {p256dh, auth}}, an https:// endpoint). One with another key is refused, so a client whose stored subscription names an old key subscribes again. A subscription belongs to one paired device; subscribing again replaces it, web_push_unsubscribe and revoking the device remove it, and so does the push service answering 404 or 410. Each push is one aes128gcm record (RFC 8291) signed by the key (RFC 8292, sub from subject or the public_url’s origin), TTL a day, Urgency: high for questions and alerts. It is never retried. Its plaintext is JSON: {title, subtitle, body, topic, message, question, alert, badge}, topic the chat (null the root chat), message its message, and alert the alert’s sound or null.