> ## Documentation Index
> Fetch the complete documentation index at: https://comis-feature-matrix-channel.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.
# Media
> Image analysis, text-to-speech, audio transcription, video description, and document extraction
**What it does:** Gives agents the ability to process images, audio, video, and documents -- analyze, transcribe, synthesize, generate, and extract.
**Who it's for:** Agents living in group chats where users share photos, voice notes, documents, and video clips. Instead of ignoring media content, your agent can understand and respond to it.
## Quick Reference
| Tool | What It Does | Provider Categories |
| ------------------ | ------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `image_analyze` | Analyze images using AI vision (file, URL, base64, or attachment URL) | Vision (follows the agent's main vision-capable model -- no separate key needed; else the Anthropic / OpenAI / Google registry) |
| `image_generate` | Generate an image from a text prompt and deliver it to the current channel | Image gen (follows the agent's main provider by default; a Codex/ChatGPT-login main needs no image key; OpenRouter / fal opt-in) |
| `video_generate` | Generate a short video from a text prompt and deliver it to the current channel | Video gen (follows the agent's main provider by default — Google→Veo, xAI→Grok Imagine, reusing its key; or explicit `fal` + `FAL_KEY`) |
| `tts_synthesize` | Generate spoken audio from text -- returns a workspace file path | TTS (OpenAI, ElevenLabs, Edge) |
| `transcribe_audio` | Convert speech to text | STT (OpenAI Whisper, Groq, Deepgram) |
| `describe_video` | Describe raw video content using vision | Vision (Gemini-only raw video) |
| `extract_document` | Extract text from PDFs, CSVs, and other documents | None (local extraction) |
For per-provider setup, model defaults, and configuration, see the [Media & Voice](/media/index) section.
## Tool Details
Use `image_analyze` to have your agent look at an image and describe what it sees, answer questions about the content, or extract information like text in a screenshot.
The tool supports multiple ways to provide an image:
| Parameter | Type | Required | Description |
| ---------------- | ------ | -------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| `action` | string | Yes | Always `"analyze"` |
| `source_type` | string | No | How the image is provided: `"file"`, `"url"`, or `"base64"` (required unless `attachment_url` is used) |
| `source` | string | No | The image data -- a file path, URL, or base64-encoded string (required unless `attachment_url` is used) |
| `prompt` | string | No | A specific question about the image (e.g., "What text is visible in this screenshot?"). Defaults to a general description. |
| `attachment_url` | string | No | Platform attachment URL from a message hint (e.g., `tg-file://...`, `discord://...`). When provided, overrides `source_type`/`source`. |
| `mime_type` | string | No | MIME type for base64 input (auto-detected if omitted) |
**Source types explained:**
* **file** -- A path to an image file on the server (e.g., `/tmp/screenshot.png`)
* **url** -- A publicly accessible image URL
* **base64** -- Raw image data encoded as a base64 string
For images attached to chat messages, use the `attachment_url` parameter instead of `source_type`/`source`. The `attachment_url` is provided in the message hint and resolves platform-specific attachment URLs automatically.
**Example usage:**
When a user sends a photo in a chat, the agent can analyze it:
```
"Analyze this screenshot and tell me what application is shown."
```
The agent calls `image_analyze` with `attachment_url` from the message hint, then responds with a description of the image content.
Use the `prompt` parameter to ask specific questions about images. Without a prompt, the agent generates a general description. With a prompt like "What error message is shown?", the agent focuses on extracting that specific information.
**Vision follows the agent's main provider.** When the agent's main model is itself vision-capable (Claude, GPT-4o, Gemini, and other models that accept image input), `image_analyze` routes the image through **that main model**, reusing the agent's existing credentials -- so **no separate vision key is required** in that case. The model that reasons for the agent is the model that "sees" the image, over one completion call.
The resolver tries the tiers in this locked order and logs which tier served (the `path` field on the completion log):
1. **main-vision** -- the agent's main model, when it accepts image input (reuses the main provider's key / Codex OAuth profile). No separate vision key needed.
2. **registry** -- the independent vision registry's own OpenAI -> Anthropic -> Google chain (its own configured keys; unchanged for current setups). Used when the main model is not vision-capable, when an explicit `defaultProvider` is set (see below), or as the fallback if a main-vision attempt fails at runtime.
3. **honest "unavailable"** -- when no tier can serve, the tool reports an unavailable result with an error kind and a hint naming the knob to set, never a crash or a misroute to the wrong provider.
An explicit `integrations.media.vision.defaultProvider` in `config.yaml` **overrides** main-first (explicit operator config wins) -- the registry tier serves that provider directly.
Use `tts_synthesize` to convert text into spoken audio. The tool returns a workspace file path (under `media/tts/`) along with the MIME type and size. The agent must then send that file via the `message` tool's `attach` action to deliver it to a chat -- TTS itself does not auto-deliver.
| Parameter | Type | Required | Description |
| --------- | ------ | -------- | ------------------------------------------------------------------------------------------------------------------ |
| `action` | string | Yes | Always `"synthesize"` |
| `text` | string | Yes | The text to speak |
| `voice` | string | No | Voice identifier (provider-specific, available voices depend on your configured provider) |
| `format` | string | No | Audio format: `"mp3"`, `"opus"`, or `"wav"` (default: resolved from config and channel; Telegram defaults to opus) |
The TTS provider (OpenAI, ElevenLabs, or Edge TTS) is determined by your `config.yaml` -- there is no per-call `provider` parameter.
**Supported providers (configured in config.yaml):**
* **OpenAI** -- High-quality voices with natural intonation. Requires an OpenAI API key.
* **ElevenLabs** -- Wide selection of realistic voices with emotion control. Requires an ElevenLabs API key.
* **Edge TTS** -- Free text-to-speech using Microsoft Edge's built-in voices. No API key required.
**Returns:** `{ filePath, mimeType, sizeBytes }`.
**Example two-step usage:**
1. `tts_synthesize` with the summary text -> obtain `filePath`.
2. `message` action `attach` with `attachment_url: file://{filePath}`, `attachment_type: audio` to deliver the audio file.
Use `transcribe_audio` to convert an audio or voice message into written text. This is useful for processing voice messages that users send in chat.
| Parameter | Type | Required | Description |
| ---------------- | ------ | -------- | ----------------------------------------------------------------------------------------------------- |
| `attachment_url` | string | Yes | The URL of the audio file to transcribe (from a message hint, e.g., `tg-file://...`, `discord://...`) |
| `language` | string | No | BCP-47 language hint to improve transcription accuracy (e.g., `"en"`, `"he"`, `"es"`) |
**Supported providers:**
* **OpenAI Whisper** -- High accuracy across many languages
* **Groq** -- Fast transcription with Whisper models
* **Deepgram** -- Real-time transcription with speaker detection
The provider used depends on your configuration. The tool returns the transcribed text, which the agent can then use in its response.
**Supported audio formats:**
Most common audio formats are supported, including MP3, WAV, OGG, M4A, and WEBM. Voice messages from chat platforms (Discord, Telegram, WhatsApp) are automatically handled.
**Example usage:**
When a user sends a voice message, the agent automatically transcribes it and responds to the spoken content. This works especially well in chat platforms where voice messages are common -- the agent can participate in voice-message conversations by reading the transcribed text and responding in text or with a TTS audio reply.
Use `describe_video` to have your agent watch a video and describe what happens in it. The tool sends the raw video to a Gemini vision model (the only provider that accepts raw video natively in this release -- there is no frame extraction); see the routing note below.
| Parameter | Type | Required | Description |
| ---------------- | ------ | -------- | ------------------------------------------------------------------------------------------ |
| `attachment_url` | string | Yes | The URL of the video to describe (e.g., `tg-file://...`, `discord://...`) |
| `prompt` | string | No | Custom analysis prompt to guide the description (defaults to a generic description prompt) |
The tool sends the raw video to a video-capable vision model and returns a coherent description of the content. This is useful for understanding short clips, tutorials, or screen recordings shared in chat.
**How it works:**
1. The video is downloaded from the provided URL.
2. The raw video buffer is sent to a **video-capable** vision provider for analysis.
3. The provider returns a description of the video content.
**Raw video is Gemini-only.** Unlike `image_analyze`, `describe_video` does **not** route through an arbitrary main model -- only Gemini accepts the raw video buffer (there is no frame extraction in this release). So `describe_video` goes straight to the **gemini-video** tier (via the vision registry's Google provider); if no video-capable provider is configured, the tool reports an honest "unavailable" result with a hint to configure Gemini, never a crash. A non-Gemini main model does not change this -- the main-provider vision path applies to `image_analyze`, not raw video.
**Example usage:**
When a user shares a video clip, the agent can describe what is happening:
```
"What's going on in this video?"
```
The agent watches the video and responds with a description like: "The video shows a step-by-step tutorial for configuring a Discord bot, starting with the developer portal and ending with the bot joining a server."
Use `extract_document` to pull readable text from documents. This is useful for processing PDFs, spreadsheets, and other files that users share in chat.
| Parameter | Type | Required | Description |
| ---------------- | ------ | -------- | --------------------------------------------------------- |
| `attachment_url` | string | Yes | The URL of the document to extract text from |
| `max_chars` | number | No | Maximum number of characters to extract from the document |
**Supported formats:**
* **PDF** -- Extracts text from PDF documents
* **CSV** -- Reads comma-separated data
* **TXT** -- Plain text files
* Other text-based formats
The tool returns the extracted text content, which the agent can then summarize, answer questions about, or process further.
**Example usage:**
```
"Read this PDF and give me a summary of the key points."
```
The agent extracts the text from the PDF, then uses that text to generate a summary. For spreadsheets (CSV), the data is returned in a structured format that the agent can analyze, filter, or summarize.
Document extraction works best with text-based PDFs. Scanned documents (images of text) may require the `image_analyze` tool instead, since the content is stored as images rather than selectable text.
Use `image_generate` to create images from text descriptions. The generated image is delivered directly to the current channel via the daemon.
By default (`integrations.media.imageGeneration.provider: auto`) image generation **follows the agent's main provider and reuses its credentials** -- no image-specific key is needed when the main provider can generate images. If the agent's main provider cannot generate images (e.g. Anthropic) and no explicit `provider` is set, the tool returns an honest "unavailable" with a hint to set `integrations.media.imageGeneration.provider` -- it never silently routes to a different paid provider. To opt in to OpenRouter, set `provider: openrouter` and supply `OPENROUTER_API_KEY`.
**Codex (ChatGPT-login) agents generate images with no API key.** When the agent's main provider is `openai-codex` (`provider: auto`), or you set `provider: openai-codex` explicitly, `image_generate` **reuses the agent's existing ChatGPT/Codex OAuth credentials** -- there is no image-specific API key to set. The OAuth bearer is resolved per call, so an expired token refreshes automatically; if the agent is not logged in, the tool returns an honest "unavailable" with a `comis auth login --provider openai-codex` hint (it never silently routes elsewhere).
**Native OpenAI and Google (key-auth) follow-main paths are wired.** A key-auth `openai` main (with `OPENAI_API_KEY`) generates via the **OpenAI Images API** (default `gpt-image-1`), and a `google` main (with `GOOGLE_API_KEY`) generates via **Gemini** (default `gemini-2.5-flash-image`) -- both under `provider: auto`, reusing the same key the main provider uses (no image-specific key). You can also pin them explicitly with `provider: openai` / `provider: google`.
| Provider (resolved main) | Images path | `provider` value | Default model | Credential |
| ------------------------------------------- | -------------------------- | ------------------------ | ------------------------------ | --------------------------------------------- |
| `openai` | OpenAI Images API | `auto` or `openai` | `gpt-image-1` | `OPENAI_API_KEY` |
| `openai-codex` | ChatGPT/Codex (Responses) | `auto` or `openai-codex` | `gpt-image-1` | ChatGPT/Codex OAuth (no key) |
| `google` / `google-vertex` | Gemini (`generateContent`) | `auto` or `google` | `gemini-2.5-flash-image` | `GOOGLE_API_KEY` |
| `openrouter` | OpenRouter (built-in) | `auto` or `openrouter` | `black-forest-labs/flux.2-pro` | `OPENROUTER_API_KEY` |
| *(explicit only)* | fal.ai | `fal` | provider preset | `FAL_KEY` |
| `anthropic` and other image-incapable mains | — | — | — | honest "unavailable" (never a silent reroute) |
| Parameter | Type | Required | Description |
| ----------------- | ------ | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `prompt` | string | Yes | Text description of the image to generate |
| `size` | string | No | Provider-specific size. fal.ai uses presets (`square_hd`, `landscape_16_9`); OpenAI uses pixel dimensions (`1024x1024`, `1792x1024`). Omit for the provider default. |
| `model` | string | No | Override the provider's default image model (e.g. `gpt-image-1`, `gemini-2.5-flash-image`). Rejected with an actionable hint listing the valid models if it is not valid for the active provider. Omit to use the default. |
| `reference_image` | string | No | A workspace file path, URL, or data-URI of a reference image for edit/img2img. The OpenAI path routes it to `images.edit`; the Google path adds it as an `inlineData` part. Omit for text-to-image. |
A `reference_image` is resolved safely: a workspace **file path** is confined to the agent's workspace (path-traversal blocked), a **URL** passes an SSRF guard before any fetch (and is size-capped), and a **data-URI/base64** is decoded in place.
The daemon-side handler applies rate limiting, safety checking, model validation, and provider execution before delivering the generated image to the current channel.
**Delivery, persistence, and degradation.** A generated image is first persisted to the agent's confined workspace under `~/.comis/workspace/media/photos/` (a UUID filename, size-capped) and then delivered from that durable path -- never an ephemeral OS temp file. Delivery is **capability-driven**, not a hardcoded channel list: a channel whose adapter implements attachment sending receives the persisted file directly; a channel without it (currently only **IRC**) degrades to a bounded base64 result returned over RPC. The same base64 fallback is used when the request has no originating channel, or if persistence or channel delivery fails -- so a generation is never lost, it is just delivered by the best means the channel supports.
**Cost ceiling.** Beyond the `maxPerHour` count limit, an optional per-agent **USD** ceiling, `integrations.media.imageGeneration.maxCostPerHourUsd`, bounds spend: once an agent's accumulated image-generation cost in the current hour reaches the ceiling, further requests are blocked with `errorKind: quota_exceeded` (and a hint naming the key) **before** the provider is called. Unset means no cost ceiling (the count limit still applies). See [config-yaml.mdx](/reference/config-yaml#integrations).
**Observability.** Each generation logs a single INFO completion line carrying the executing image provider, the agent's main provider, the model, `durationMs`, `costUsd`, `sizeBytes`, and the mime type; every failure branch logs an `errorKind` + an actionable `hint` (never the raw provider message, a key, or a bearer). The image turn is reconstructable from observability: `comis explain ` surfaces the image block (provider / model / `costUsd` / outcome / delivered) from the session trajectory, and the cost also flows into the per-session cost rollups (`comis fleet`).
When following a Codex main provider, Comis identifies itself to the ChatGPT backend as the official Codex client -- it sends the same first-party client headers (`originator: codex_cli_rs`, a Codex-shaped `User-Agent`, a `ChatGPT-Account-ID` decoded from your token, and per-request `session-id`/`x-client-request-id` identifiers) that the official Codex CLI sends. This is a **first-party-client compatibility shim** using **your own** ChatGPT/Codex OAuth credentials against the **same endpoint** the official Codex CLI uses -- so Cloudflare does not 403 requests from non-residential IPs. It is authorized reuse of your own credentials, **not detection evasion**. If a future change to the ChatGPT backend rejects this image path, set `integrations.media.imageGeneration.provider: openrouter` (with `OPENROUTER_API_KEY`) as the explicit fallback.
Use `video_generate` to create a short video clip from a text prompt. Every backend -- native **Google Veo**, **xAI Grok Imagine**, and the **FAL** queue API -- runs through one unified submit -> poll -> download contract.
**Async by default (the render outlives the turn).** A video render takes 30 s-5 min -- longer than a single agent turn and longer than the `promptTimeout`. So `video_generate` does **not** block: it submits the job, persists a durable job record, and returns a **job handle** (`{ jobId, state: "submitted", estimatedCostUsd }`) promptly. A background poller then drives the render to completion **independent of the originating turn and across a daemon restart**, downloads the clip before the (expiring) provider URL dies, persists it, and **announces the finished video to the originating channel** on its own. Use the [`video_status`](#video_status) tool with the returned `jobId` to check progress in the meantime. Delivery is **at-least-once**: a clip is delivered once it completes; if the daemon restarts mid-render it resumes from the persisted job and still delivers -- so a rare, bounded duplicate is possible, but a render is never lost and never redelivered indefinitely.
**Provider following (`integrations.media.videoGeneration.provider: auto`, the default).** Video generation follows the agent's main provider and reuses its credentials, mirroring image generation: a `google` main resolves to **Google Veo** and an `xai` main resolves to **xAI Grok Imagine**, each with the key the agent already uses (`GOOGLE_API_KEY` / `XAI_API_KEY`) -- no video-specific key. Where the main provider has no video API (e.g. `openai`, `anthropic`), the tool returns an honest "unavailable" with a hint to set `integrations.media.videoGeneration.provider` (e.g. `fal` + `FAL_KEY`) -- it never silently routes to a different paid provider.
Under `provider: auto` (the default), a **`google`** main generates via **Google Veo** (default `veo-3.0-fast-generate-001`, GA -- generates audio by default) reusing `GOOGLE_API_KEY`, and an **`xai`** main generates via **xAI Grok Imagine** (`grok-imagine-video`, 480p/720p) reusing `XAI_API_KEY` (or a SuperGrok login) -- no video-specific key. Where the main provider has no video API, the tool still returns an honest "unavailable" naming the opt-in path. The native-Veo path also has a **FAL-hosted fallback**: set `provider: fal` + `model: fal-ai/veo3.1` to render Veo through the FAL queue instead (it needs a second credential, `FAL_KEY`, so it is not the follow-main default).
| Parameter | Type | Required | Description |
| ----------------- | ------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `prompt` | string | Yes | Text description of the video to generate |
| `duration` | number | No | Clip length in seconds (provider-validated; default from `defaultDurationSecs`, e.g. 8) |
| `aspect_ratio` | string | No | `16:9`, `9:16`, `1:1` (provider-validated; default from `defaultAspectRatio`) |
| `resolution` | string | No | `720p`, `1080p`, `4k` (provider-validated; default from `defaultResolution`) |
| `audio` | boolean | No | Generate audio when the backend supports it (raises the cost estimate) |
| `negative_prompt` | string | No | What to avoid (provider-dependent) |
| `seed` | number | No | Deterministic seed (provider-dependent) |
| `image_url` | string | No | A workspace file path, URL, or data-URI of a source image for **image-to-video**. When present, the tool renders an image-to-video clip on backends that support it -- **FAL** swaps to its image-to-video endpoint, and **Veo**/**Grok** add the source image to the same model. SSRF-guarded + workspace-confined. Omit for text-to-video. |
| `model` | string | No | Override the backend's default video model/endpoint (e.g. `fal-ai/veo3.1/fast`). Omit to use the default. |
**Per-backend options (validated against the active backend).** The clip-shape params are validated against the **active backend's capability matrix** *before* the render is submitted: an unsupported value (a resolution the backend cannot produce, a duration outside its range, or `image_url` on a text-to-video-only model) is rejected up front with a hint listing the valid set for the current backend -- it is never silently routed to a wrong value or surfaced as a provider error. The concrete per-backend options today are:
| Backend | Durations | Resolutions | Audio | Image-to-video |
| ------------------------------------- | ------------------------------------------ | ---------------------------------------------------------------- | ----- | ---------------------------------------------- |
| **FAL** (`fal-ai/veo3.1/fast`) | `4` / `6` / `8` s (snapped to the nearest) | `720p` / `1080p` / `4k` | yes | yes (swaps to the FAL image-to-video endpoint) |
| **Veo** (`veo-3.0-fast-generate-001`) | `4` / `6` / `8` s (snapped) | `720p` / `1080p` / `4k` (`1080p`/`4k` require an `8` s duration) | yes | yes |
| **Grok** (`grok-imagine-video`) | `1`-`15` s (clamped) | `480p` / `720p` only | no | yes |
Because the active backend follows the agent's main provider (or `integrations.media.videoGeneration.provider`), the `video_generate` tool **description the agent sees is built at daemon startup from the active backend's real options** -- so the agent is told the active provider's durations/resolutions/aspect ratios and whether image-to-video is supported up front, rather than a static superset. Changing `provider` therefore needs a daemon restart for the description to refresh (the same boot-bound contract as provider/credential selection).
Image-to-video currently takes a **single** source image (`image_url`). Supplying multiple reference images (e.g. distinct first-frame / last-frame / style-reference roles) is a planned follow-up and is not yet available.
**Cost ceiling (estimate-first).** Video is dollars-per-clip and is **already rendering once submitted**, so unlike image generation the per-agent **USD** ceiling, `integrations.media.videoGeneration.maxCostPerHourUsd`, is gated on a **worst-case pre-submit estimate** (duration x per-second rate, with audio/4k as the upper bound): if the estimate would push the agent's accumulated hourly spend over the ceiling, the request is blocked with `errorKind: quota_exceeded` (and a hint naming the key) **before** the provider is called. The `maxPerHour` count limit is retained and orthogonal. Unset means no cost ceiling.
**Delivery, persistence, and degradation (in the background poller).** When the render completes, the poller downloads the clip to a buffer and persists it to the agent's confined workspace under `~/.comis/workspace/media/videos/` (a UUID filename, size-capped) **before** any delivery decision -- an expiring provider URL cannot orphan it. Delivery is **capability-driven**: a channel whose adapter implements attachment sending receives the persisted file directly (as a `video` attachment); a channel without it (currently only **IRC**) degrades to a notice + the persisted path. Because delivery happens off-turn from the persisted job, the finished clip reaches the channel that requested it even after the turn has ended (and after a daemon restart). The persisted file is not deleted after delivery, so generated clips survive for later inspection.
**Oversized-clip degrade (never silently dropped).** A 4K/long clip can exceed a channel's video-upload limit. Rather than drop it, the poller checks the persisted clip's size against a **per-channel** limit before sending and degrades gracefully -- it is never routed through the silent "attachment too large -> fallback text" path. On a channel that renders URLs (Discord, Slack, Telegram, LINE, WhatsApp, Signal, iMessage, Email) the poller sends a **link** message to the retained provider URL when one is available; otherwise (or on a notice-only channel) it sends a **notice** message naming the saved workspace path. Either way the clip stays persisted and recoverable, and the message always carries the saved path. The per-channel limit follows each platform's documented bot/upload cap (e.g. WhatsApp \~16 MB, Discord \~25 MB, Telegram \~50 MB, Slack/Signal/iMessage \~100 MB, LINE \~200 MB) and is overridable. IRC, which has no attachment surface at all, degrades to a notice via the same capability gate.
**Observability.** When the render completes the poller logs a single INFO completion line carrying the executing video provider, the model, `costUsd`, `sizeBytes`, `durationMs`, and the mime type; every failure branch (submit failure, a failed render, or a poll timeout) logs an `errorKind` + an actionable `hint` (never the raw provider message, a key, a bearer, or the Veo keyed download URL -- a raw provider/channel error message is scrubbed before it rides a log line). For a provider-native render (Veo/Grok) the poller preserves the **specific** classified failure kind off-turn -- a safety-policy block is logged (and persisted to the job's `error`) as `content_blocked`, a quota/credits limit as `quota_exceeded`, and so on -- rather than collapsing every terminal failure to a generic empty result, so a content-policy block is distinguishable from a transient empty render in the logs alone. Because the completion runs off-turn, the poller stamps the originating `traceId` (captured at submit) on its log lines so the later completion stitches back to the request.
**`comis explain` reconstruction (incl. background completion).** A video turn is reconstructable from observability: `comis explain ` surfaces a `videoGenerated` block (provider, model, `jobId`, `estimatedCostUsd`, the reconciled `costUsd`, `outcome`, `delivered`) from the session trajectory -- **including a job that completed in the background after the originating turn ended**: the submit record (written in-turn) and the off-turn completion record are stitched by the carried `jobId`/`traceId` on the originating `session_key` (the column added to the [`video_jobs`](/operations/data-directory#video-job-store-async-render-lifecycle) store for exactly this). The reconciled cost also flows into the per-session cost rollups (`comis fleet`). **Cost provenance is per-backend:** FAL and Veo report no per-call actual cost, so the reconstructed cost is the pre-submit **estimate**; xAI Grok reports an **actual** cost (from its usage ticks) that is reconciled in. (A live `comis explain` against a real background-completed render is an operator check; the deterministic reconstruction path is the binding contract.)
Use `video_status` to check the status of a video-generation job by the `jobId` that [`video_generate`](#video_generate) returned. Because a render outlives the turn (and survives a daemon restart), `video_status` is how the agent polls progress while the background poller drives the job to completion -- the finished clip is then delivered to the channel automatically (no `video_status` call is required for delivery; it is for visibility).
| Parameter | Type | Required | Description |
| --------- | ------ | -------- | -------------------------------------------- |
| `job_id` | string | Yes | The job handle returned by `video_generate`. |
The response reports the job's durable state:
| Field | Type | When present |
| ----------- | ------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `state` | `"pending"` \| `"done"` \| `"failed"` | Always. |
| `progress` | number (0-1) | While `pending`, when the provider reports it. |
| `mediaPath` | string | When `done` -- the persisted workspace path of the delivered clip. |
| `costUsd` | number | When `done` -- the reconciled actual cost. |
| `error` | string | When `failed` -- an actionable, secret-free hint describing the failure and what to do (e.g. "xAI blocked the prompt by moderation. Revise the prompt and retry." for a content block, or a timeout hint naming the `timeoutMs` knob). The matching machine-readable failure kind (`content_blocked`, `quota_exceeded`, `auth_required`, `job_timeout`, `empty_response`, ...) is carried on the off-turn poller's `errorKind` log field, not this string. |
**Agent-scoped.** `video_status` is scoped to the **calling agent**: a `jobId` that belongs to a *different* agent returns a not-found result (`state: "failed"`), never another agent's media path or cost. In a multi-agent daemon an agent can only see its own jobs.
## Common Workflows
Here are some typical ways agents use media tools together:
* **Voice message handling** -- When a user sends a voice note, the agent uses `transcribe_audio` to convert it to text, processes the request, and optionally uses `tts_synthesize` to reply with audio.
* **Document Q\&A** -- A user shares a PDF report. The agent uses `extract_document` to read it, then answers questions about the content.
* **Image moderation** -- In a group chat, the agent uses `image_analyze` to check shared images for content that violates community guidelines.
* **Accessibility** -- The agent uses `describe_video` and `image_analyze` to provide text descriptions of visual content for users who need them.
## Provider Configuration
Media tools route to providers based on the credentials present in your secret store and the providers listed in `config.yaml`. None of the tools accept a `provider` parameter at call time -- selection is config-driven, not per-call.
| Capability | Available providers | Auto-selection order | Required env keys |
| ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Vision -- `image_analyze` | **Follows the agent's main provider** when it is vision-capable (Claude / GPT-4o / Gemini and other image-accepting models), reusing its key; else the independent vision registry (OpenAI / Anthropic / Google) | main-vision -> registry (OpenAI -> Anthropic -> Google, first available) -> honest unavailable; an explicit `defaultProvider` overrides main-first | **None in the main-vision case** (reuses the main provider's key / Codex OAuth profile). The registry fallback uses `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GOOGLE_API_KEY` (at least one). |
| Vision -- `describe_video` (raw video) | **Gemini only** (raw video buffer; no frame extraction this release) | gemini-video -> honest unavailable | `GOOGLE_API_KEY` (the vision registry's Google provider) |
| TTS (`tts_synthesize`) | OpenAI TTS, ElevenLabs, Edge | Configured order; `auto-tts` heuristic decides when to use voice | `OPENAI_API_KEY`, `ELEVENLABS_API_KEY` (Edge is free) |
| STT (`transcribe_audio`) | OpenAI Whisper (gpt-4o-mini-transcribe), Groq Whisper, Deepgram nova-3 | Fallback chain in declared order | `OPENAI_API_KEY`, `GROQ_API_KEY`, `DEEPGRAM_API_KEY` (at least one) |
| Image generation (`image_generate`) | Follows the agent's main provider (`auto`): native **OpenAI** (`gpt-image-1`), **Google Gemini** (`gemini-2.5-flash-image`), **Codex** (ChatGPT-login), and **OpenRouter**; explicit `openrouter` / `openai-codex` / `openai` / `google` / `fal` | `auto` = the agent's main provider (reuses its key, or OAuth bearer for a Codex main); else the explicit `provider` | None in the `auto` case (reuses the main provider's key): `OPENAI_API_KEY` for an `openai` main, `GOOGLE_API_KEY` for a `google` main, the **OAuth bearer** for a Codex main (no image key). `OPENROUTER_API_KEY` for the OpenRouter opt-in; `FAL_KEY` for explicit `fal`. Honest "unavailable" (not silent) if the main provider can't generate images and no `provider` is set. |
| Video generation (`video_generate`) | Follows the agent's main provider (`auto`): native **Google Veo** (`veo-3.0-fast-generate-001`) for a `google` main, native **xAI Grok Imagine** (`grok-imagine-video`) for an `xai` main; explicit **`fal`** (FAL queue, `fal-ai/veo3.1/fast`; also the FAL-hosted-Veo fallback via `model: fal-ai/veo3.1`) | `auto` = the agent's main provider (reuses its key); else the explicit `provider` (`fal` / `google` / `xai`) | `FAL_KEY` for explicit `fal`; in the `auto` case reuses the main provider's key (`GOOGLE_API_KEY` for Veo, `XAI_API_KEY` for Grok). Honest "unavailable" (not silent) if the main provider can't generate video and no `provider` is set. |
| Document extraction (`extract_document`) | Local pdf.js + CSV / text decoders | n/a | None (uses FFmpeg for some audio metadata) |
### Cost notes
Media calls bill against your model providers, not Comis itself. Rough rules of thumb:
* **Vision** is significantly more expensive per call than text generation -- a single high-res image can cost 1-5K tokens of input. Use `prompt` to focus the analysis and avoid re-analyzing the same image across turns.
* **STT** is generally cheap (Groq's whisper-large-v3-turbo is the lowest-latency / lowest-cost option in the fallback chain).
* **TTS** is priced per character. ElevenLabs is the priciest tier; Edge TTS is free but voice quality is more limited.
* **Image generation** has the highest per-call cost. The image-gen rate limiter is conservative by default (\~1 req/min on OpenAI, \~1 req/sec on FAL), and you can additionally cap **spend** per agent/hour with `integrations.media.imageGeneration.maxCostPerHourUsd` (requests over the ceiling are blocked with `quota_exceeded`). Each generation's `costUsd` is recorded in the session trajectory (`comis explain`) and the cost rollups (`comis fleet`).
Comis applies graceful degradation: if no credentials are configured for a capability, the corresponding tool is omitted from the agent's tool catalog rather than failing at call time. For `image_analyze`, vision follows the agent's main provider first (reusing its key when the main model is vision-capable -- no separate vision key needed), then the independent vision registry, then an honest "unavailable" result; an explicit `integrations.media.vision.defaultProvider` overrides main-first.
See the [Media & Voice](/media/index) section for detailed provider setup, the [Vision](/media/vision) page for image flow details, and [Voice](/media/voice) for TTS/STT options.
All media tools process content on your server. Files are not sent to third parties beyond the configured AI provider for analysis.
## Enabling Media Tools
Media tools are enabled by default in the `full` tool policy profile. If you are using a restricted profile (like `minimal` or `coding`), you can enable specific media tools by adding them to your agent's `allow` list:
```yaml theme={null}
skills:
toolPolicy:
profile: minimal
allow:
- image_analyze
- transcribe_audio
```
See [Tool Policy](/skills/tool-policy) for more details on controlling which tools your agents can access.
## Related
Detailed media provider setup and configuration
How AI vision processing works
Text-to-speech and transcription configuration
See all available agent tools