Voice agent QA review board for your Hume EVI calls

By General Input

Open one board each morning, see which voice calls went badly, replay the exact moment the caller got frustrated, and file the fix.

Integrations

  • Hume
  • Linear
  • Slack Bot

Type

App

Categories

  • Customer Support
  • Operations

Build me a QA review board for our Hume EVI voice deployment. Our support quality lead should open this every morning instead of clicking through the Hume portal. This is a browsing and drill down surface for a human reviewer, not an automation: nothing in it runs on a schedule, nothing escalates on its own, and the only work that happens is work the reviewer presses a button to start.

The main view is a table of recent EVI conversations. Pull them with the Hume List Chats operation, and use List Chat Groups so a conversation that spans several resumed sessions is recognizable as one thread rather than showing up as unrelated rows. Each row shows the start time, the call duration, the number of turns, the EVI config that handled the call, and a rolled up expression signal. Resolve config identifiers to readable names using List EVI Configs rather than showing raw ids. Default the sort so the roughest calls are at the top, and let the reviewer re-sort by any column.

The rolled up expression signal is the heart of the queue and should be computed from the emotion features that Hume attaches to each transcript event. Read the chat events for a call, look at the expression measures on the caller's turns, and roll the negative ones such as frustration, anger, confusion, distress and disappointment into a single score for the call, with a peak value as well as an average so a call that was mostly fine but had one terrible moment still surfaces. Because this requires fetching events per call, do not recompute it on every page load. Compute it lazily in the background as rows come into view, cache the result in the app's own storage keyed by chat id, and show a quiet placeholder on rows not scored yet. A call's events never change once it has ended, so a cached score never needs invalidating.

Clicking a row opens the call detail. Build a transcript timeline from List Chat Events, showing each user and assistant turn in order with its timestamp. Next to every turn, display the vocal expression measures attached to that turn, showing the few strongest emotions rather than dumping the full measure set, and visually mark the turns where negative expression spikes relative to the rest of the call. The goal is that a reviewer can scroll the timeline and see exactly where the caller got frustrated, and what the assistant said immediately before and after.

The detail view also has an audio player fed by Get Chat Audio. Bake in that Hume chat audio is asynchronous: that operation returns a status of QUEUED, IN_PROGRESS, COMPLETE, ERROR or CANCELLED, and a signed audio URL is only present when the status is COMPLETE. So the player must show a pending state for QUEUED and IN_PROGRESS, with a re-check so the reviewer can see when it becomes ready, and only offer real playback once the status is COMPLETE and a signed URL exists. For ERROR and CANCELLED, show a plain message saying the recording is not available for this call instead of leaving a spinner running forever. Treat the signed URL as short lived: fetch it when the reviewer opens the call rather than storing it and serving a stale link later.

Add a "Review this call" button on the detail view that kicks off a background agent for that one call. The agent reads that call's chat events including the per turn expression measures, pinpoints the specific turn where sentiment turned, judges whether the assistant recovered the conversation afterward or made it worse, and writes back a short QA verdict of a few sentences plus one concrete suggested prompt fix for the EVI config that handled the call. Its output lands in the app's storage against that chat and displays on the queue row, so the reviewer can see at a glance which calls have been looked at and what the verdict was. This runs on demand only, one call at a time, when the button is pressed. It never runs on a schedule and it never files or posts anything by itself.

From a reviewed call, give the reviewer two follow up actions. "File issue" opens a Linear issue using Create Issue, with a title naming the problem, and a description containing the QA verdict, the suggested prompt fix, and the excerpt of the transcript around the turn where sentiment turned, so the issue is readable without opening this app. Let the reviewer pick the destination team, populated from List Teams. "Share" posts the call summary and verdict into a Slack channel using the Slack Bot Send a Message operation, with the channel chosen from List Channels. After either action, store the resulting issue link or the fact that it was shared on the call, and show that on the row so the same call does not get filed twice.

Make the queue drainable. Let the reviewer mark a call as reviewed, filter to unreviewed only, and filter by EVI config and by date range. Persist reviewed state, the agent's verdict and suggested fix, and any filed issue link in the app's own storage keyed by chat id, so all of that survives across sessions and is never lost just because the call is old. Default the board to the last seven days of unreviewed calls, since that is what someone opening it in the morning actually wants.

One API detail to handle correctly: Hume list endpoints are zero indexed and paginated with a page size of at most 100, returning a page number and total page count. Page through properly when loading the queue and when reading a long call's events, rather than assuming the first page is everything.

Related prompts

Explore more prompts
A brand asset library your marketing team actually searchesTurn Mailjet email clicks into ranked HubSpot follow-upsClean out the Looker dashboards and Looks nobody opensLiveKit live operations console for room moderationWake up dormant Keap leads with a researched reasonLiveChat coverage board for planning next week's shiftsPhone routing control panel for LiveKit voice agentsLinkedIn Ads budget pacing dashboard for every client accountGive your team Looker numbers without buying more seatsPause marketing emails to escalated customers, then restore them