Mystic Horizon / AI assistants · Voice & follow-up
A conversation is only the beginning.
I built a voice-agent prototype that keeps a record of conversations and follow-up actions. Its operator dashboard makes the conversation, stored context, and unfinished work available for inspection.
The situation
A useful conversation can leave behind scattered details and a promise that nobody follows up. An agent needs a record of the interaction, a way to recover relevant context, and an explicit account of what it intends to do next.
Useful when exploring an assistant that needs to remember context, track a commitment, and expose what happened after a conversation.
One concrete example
One task is finished. Another stays open.
Input
A synthetic call record says a meeting was confirmed and a report discussed. Separate pending actions exist to schedule the meeting and send the report.
What happens
The integration test supplies a model judgment that the meeting task is satisfied and the report task only partly addressed. The application applies those judgments to the stored actions.
Observable result
The meeting action becomes completed. The report action stays pending with a note that it was partially addressed in the call. An unrelated proposal-review action stays unchanged.
Human review
The owner can inspect the call record and unresolved action before deciding what should happen next. Discussing the report does not mark it as sent.
What I built
I designed and implemented this personal prototype with AI-assisted development, connecting the voice interface, persistent memory, action lifecycle, and operator dashboard. The work includes provider integration, state and retrieval design, troubleshooting, and regression checks.
Where it stands
This is an independent prototype. Recorded checks do not establish live-call quality, model judgment accuracy, sustained-load performance, or production uptime. Local storage also does not mean every operation stays local: selected cloud providers process the data needed for their features.
The next useful improvement
Evaluate complete conversation-to-action scenarios with representative examples, then use the recorded state to explain failures and measure improvements.
Technical decisions, workflow, and verification
Python · aiohttp · SQLite / FTS5 / sqlite-vec · LiveKit · Twilio
Follow the workflow
Capture the interaction
Dashboard or telephone conversations become transcripts, structured facts, and commitments stored in SQLite.
Bring back useful context
Full-text and vector retrieval find relevant history for a later interaction, combining exact terms with semantic recall.
Make follow-up explicit
A scheduler evaluates pending commitments and can act, wait, cancel, or escalate. Actions have stored state that can be inspected independently of the conversation.
Keep the operator informed
The dashboard and CLI expose transcripts, people, actions, tool events, and provider readiness so the owner can investigate behavior.
Decisions and tradeoffs
Inspectable state in one database
SQLite, FTS5, and sqlite-vec keep records and retrieval close together. This is a manageable fit for a personal agent; a service shared by many customers would need a different operating model.
Choice across the speech path
LiveKit connects browser voice and optional Twilio telephony. Local speech and embedding options offer more control over processing, while adding model downloads and hardware dependencies. Cloud providers remain available.
Actions are records, not just promises
The scheduler works with persisted commitments and explicit decisions. That makes retries and investigation possible, while leaving the quality of model judgments as something to evaluate.
Explore the evidence
See the operator interface
A recorded walkthrough shows the dashboard, conversation surface, and agent state. It is a tour of the prototype rather than a performance benchmark.
Watch the recorded walkthroughInspect the implementation
The public repository maps memory, action scheduling, voice adapters, tool permissions, and dashboard behavior to their source files.
Explore the sourceReview the verification boundary
The September 4, 2026 verification record reports 630 non-benchmark tests passed, 3 skipped, and 26 benchmark cases deselected. Tests use temporary state and mocked external providers.
Read verification notes