Moose Lab / Behind the experiment
A co-pilot worth
talking to.
Bring Pip home with Moose, your voice companion on an orbital rescue. This lab follows the playtests, tool decisions, and prompt experiments behind the project.
- Missions reviewed
- —
- Benchmark runs
- —
- Reviewed command score
- —
- Models benchmarked
- —
Real measurements appear when the evaluation service is available. No illustrative scores are used.
The experiment
How does a co-pilot
earn its next version?
The goal is a companion that follows your intent, tells the truth about the game, and leaves room for conversation. Moose’s current voice connection runs through LiveKit hosted on our VPS, with Inworld speech recognition and voice synthesis around an OpenAI text model.
- 01
Play and capture
When session saving is enabled, gameplay and conversation text are saved privately for review and prompt research. The home screen shows whether saving is currently available. Mission records can preserve game facts and actual tool results across voice reconnects. Evaluation capture records no raw audio; debrief replies are optional.
- 02
Review with evidence
Code measures available latency and usage. A separate AI judge can propose turn-level labels. Missing evidence stays unknown, and an operator can add human corrections.
- 03
Experiment with instructions
GEPA evolves prompts using feedback from executions. The existing lab tests OpenAI or OpenRouter text models against the earlier crew-and-repair simulation, with a separate model proposing revisions. Those fixtures are historical; their scores are not evidence about this rescue.
- 04
Verify before release
Reserved tests measure the selected candidate. Public results identify the provider, model, suite, and sample count. Current rescue playtests and real microphone checks are still needed before changing Moose’s live instructions.
Activity is not improvement.
A busy mission and a good benchmark measure different things. Observational playtest results stay separate from controlled prompt comparisons.
A score needs context.
Text fixtures test responses and tool decisions. They cannot measure speech recognition, microphone handling, interruptions, or how the voice feels. Unknown judgments and incomplete measurements stay visible.
The process stays inspectable.
Public aggregates explain what happened. Private traces help operators understand why. Candidate prompts never deploy themselves.
Read the GEPA paper ↗ and official implementation ↗.