Ask-the-Author Archive
Turning years of chat messages and recorded interviews into a searchable, linked Q&A archive.
Runs on demand to pick up new messages.
- Role
- Sole developer: pipeline design, implementation and verification.
- Pipeline stages
- 7
- Sources
- Chat + video
- Incremental
- Cached, resumable
- Stack
- Python, NumPy, ffmpeg, Whisper (Groq API), LLM APIs (Groq, Gemini), Discord REST API
The problem
A novelist answers reader questions in a Discord channel and now and then in recorded interviews. After a few years the answers were hard to find: replies are scattered, some answers run over several messages, some were reposted by other bots, and the interviews only exist as video. The pipeline rebuilds the question and answer pairs and publishes them as one searchable archive, with a link from each answer back to the original.
Pipeline
- Collect
- Channel exportread-only REST, resumable
- Audio extractionffmpeg, 15-minute chunks
- Structure
- Reply-graph pairingreplies, follow-ups, reposts
- TranscriptionWhisper
- Speaker detectionfrom video frames
- Resolve
- LLM sortingunpaired answers → question
- LLM extractionoverlapping windows → Q&A
- Publish
- Merge & cleanmentions resolved, links kept
- Archive JSONserved by the bot
How the pairs are found
- 1.
Rules first
The author's replies point at the question they answer. Messages without a reply that follow an answer within a short window continue it. Pairs that only exist as reposts by other bots fill the remaining gaps. Whatever is left is kept with its preceding messages as context.
- 2.
A model only for what's left
For each leftover message the model sees the messages before it and picks the question it answers, or none for thanks, jokes and announcements. Answers are requested as JSON, rate limits are retried, and results are cached so a rerun only asks about new messages.
- 3.
Who's speaking, from the picture
The interviews are recordings of a Discord call, and Discord draws a green frame around the tile of whoever is speaking. Per sampled frame the detector finds thin green horizontal lines, pairs them into the top and bottom edges of a box, and classifies the speaker by the tile's colour. Each transcript segment then gets a speaker label.
- 4.
Windowed extraction
Labelled transcripts go to the model in fixed windows with an overlapping tail, so an answer that runs past the end of a window stays in one piece. A question belongs to the window it starts in.
Trade-offs
Rules before the model
- Why
- Most pairs are certain from the reply references. The model only decides the unclear cases, which keeps the cost low and the results checkable.
- Cost
- The rules are tailored to how this channel works.
Visual speaker detection instead of audio diarisation
- Why
- The recordings already carry a reliable visual signal, and a few NumPy operations per frame are cheaper than a heavy model.
- Cost
- Only works for this recording layout.
Results
The archive is served by the Aiko bot as a public, searchable page with a link from each answer back to its original message or video timestamp, and it is the only source the Aiko support assistant uses for story questions.