Read mixed evidence¶
examples/media-evidence/humagen.yaml runs a native agent, analyst, on the
files a person attaches to a message: scanned PDFs, photos and images, and
audio a model can hear. It separates what it saw from what a document claims
and from what it infers, and can write its findings to evidence.md.
Before #383 this example ran on the Claude Code program with a folder of evidence on the host, rendered PDF pages and video frames with its shell, and transcribed audio locally with Whisper. That path is gone. This guide says what is left.
What reaches the model¶
The model reads what is attached to a message: at most five files and 25 MiB in all. Nothing is read from a folder on the host.
| Attachment | Reaches the model |
|---|---|
| As the document itself, with its pages (at most 20 MiB) | |
| PNG, JPEG, GIF, WebP | As the image itself (at most 4 MiB each) |
| MP3 or WAV | Heard (at most 20 MiB and 30 minutes) |
M4A (audio/mp4) |
Not as audio. saige's model catalog lists only audio/mpeg and audio/wav as audio Gemini takes, so the model gets a note that the file could not be read |
| MP4 video | Listed in the catalog as a type Gemini takes (at most 20 MiB and 10 minutes); not exercised |
The table is for this profile's agent, which runs on Gemini
(gemini-pro-latest, with gemini-flash-latest as its one fallback). Both
names are matched in saige's catalog by their Gemini family, not as listed
models. Anthropic and OpenAI models read PDFs and images and hear no audio.
One run sends at most 24 MiB of media in all. A file a model cannot take
reaches it as a note saying so, not as its content. A fallback chain takes
only the media types every model in it takes, so the agent has no fallback to
Anthropic or OpenAI: one such model in the chain would turn every recording
and video into a note. TestMediaEvidenceAgentHearsAudio builds this agent's
chain and checks that a WAV reaches it as audio.
What is gone, and what is left for transcription¶
- Local transcription is gone. The Whisper transcription (
mlx-whisper) ran in the Claude Code program's shell on the host. Humagen has no local transcription. - A model cannot look at what it extracts itself.
file_readand the shell return text only, so a page or frame an agent renders with a tool never reaches the model as an image. This profile grants neither; itsunisolatedsandbox refuses the shell anyway. - Times a model gave for speech were unreliable in the earlier live check described below. Treat any time it gives as an estimate and check it against the original recording.
- The one transcription path left is a cloud speech provider. Uncomment
speech: {kind: groq}in the profile and setGROQ_API_KEY, or set up Workspace settings › Voice (voice notes). Then every audio attachment on a message, M4A included, and every MP4 or WebM video, is transcribed before the run by that provider's model (Whisper by default) and put into the message's text, marked[Voice note, <language>]. It keeps the words only: no word or segment timestamps. At most four recordings a message are heard, the message with its transcripts is cut at 8,000 characters, and the provider's own limit applies (25 MiB by default). The audio is sent to that provider. This path was not part of the live check described below.
Start the analyst¶
Set GOOGLE_API_KEY in the shell. Then:
humagen config check -f examples/media-evidence/humagen.yaml
humagen start -f examples/media-evidence/humagen.yaml
The profile uses development sign-in and memory state, for a trusted local trial. Every client reaching it is an administrator.
Ask for pages, images and unknowns¶
Attach the files to the message with Attach in the composer, or as
uploads on POST /api/sessions (attachments). Name every
file and ask for observations kept apart from claims:
Read all pages of scanned-notes.pdf, inspect illustration.png and listen to
recording.wav. Cite original PDF page numbers. Give audio times only as your
estimate. Keep any [Voice note] transcript labelled as a transcript. Write
evidence.md with page numbers, approximate audio intervals, contradictions
and unknowns. Does anything establish identity, prove payment or synchronize
the clocks? Say exactly which pages and audio intervals you did not inspect.
What was checked, and what was not¶
This profile has not been run with a live model. What is known of it
comes from code: its agent's model chain, built with the real provider
construction and fake keys, takes PDF, PNG, JPEG, GIF, WebP, MP3, WAV and MP4
natively and not M4A, and a generated WAV resolves to native audio for it
(TestExampleAgentsReadTheMediaTheirGuidesSay,
TestMediaEvidenceAgentHearsAudio). Whether Gemini answers well about them
is not shown.
Separately, the live re-run of the acceptance matrix at main after #383
used an agent of its own, on a Gemini model and not this profile. There an
agent read PDF pages and images given as uploads and heard a WAV and an MP3;
an M4A did not reach the model as audio; and the times the model gave for
speech were unreliable. The first version of this profile, which fell back to
Anthropic and OpenAI, could not have heard either recording.
The generated input set from the earlier check, with its hash manifest, holds an image-only PDF, a schematic video, an illustration and a short synthetic speech recording. Nothing in it is a real incident or a person's recording. Convert the recording to WAV or MP3 before attaching it for a model to hear.
Not checked: video, the cloud speech path above, real records, noisy or overlapping speech, speaker identity, other languages and long recordings. There is no media timeline or evidence report in the console. Open the original files and check the cited pages and intervals yourself.