Skip to main content
A summary looks across many evaluation runs for one agent and produces an AI-generated diagnosis of its strengths and weaknesses, a chart of how its metrics are trending, and a prioritized list of recommendations for improving it. Instead of reading run after run, you get the big picture and a clear list of what to fix first.

​
Viewing a summary

Go to Evaluations → Summaries and select an agent. Every summary shows three things:

Diagnosis

An AI narrative of the agent’s consistent strengths and recurring failure patterns.

Recommendations

Specific, actionable prompt and skill suggestions. Each carries a priority chip (High / Med / Low) and a metric tag, and can be filtered by priority or metric.

Metrics over time

A run-by-run chart with the seven quality metrics as percentage lines plus Response Time in seconds, plotted by run date.
There are two ways to view a summary: an always-on automatic one, and an on-demand one for a date range.

​
Automatic summary

With no date range selected, you see the agent’s automatic summary. VIVI regenerates it automatically after every evaluation run completes — there’s no button to press — analyzing the agent’s last 10 completed runs. At the top you’ll see ” runs analyzed”, “Generated ago”, and a stale badge if it’s more than 30 days old. While it’s working, the page shows:
  • “Generating summary…” while VIVI is analyzing.
  • “Taking longer than expected” after about five minutes.
  • A message that a summary will be generated automatically after the next run, if generation failed or there isn’t enough data yet.
  • A “Run your first evaluation” call-to-action if the agent has no runs.

​
On-demand summary (date range)

When you pick a date range, the page switches to an on-demand analysis of the runs in that window.
1

Pick a date range

Use a preset — Last 7 days, Last 14 days, or Last 30 days — or set a custom range. It analyzes the last 10 completed runs in that window.
2

Read the metrics immediately

The metrics for the selected period appear right away.
3

Generate the analysis

Click Generate diagnosis & recommendations. VIVI computes the diagnosis and recommendations in real time (this takes a few seconds).
4

Browse each run

After generating, use the run selector to view the diagnosis and recommendations for each individual run in the range.
On-demand analyses are temporary — they expire after about 5 days. You can regenerate them at any time, and your latest automatic summary is never affected.

​
Automatic vs. on-demand

Automatic

Always reflects the latest state, regenerated after each run completes. Uses the agent’s last 10 completed runs, with no date range.

On-demand

Focuses on the last 10 completed runs in a date range you choose. Metrics appear immediately; the diagnosis and recommendations are generated only when you click the button.

​
Best Practices

  • Check the automatic summary after a batch of runs to see the current state of an agent.
  • Sort recommendations by priority and tackle High items first — those move the most metrics.
  • Use the date-range view to compare a “before” and “after” window around a prompt change.
  • Pair the Diagnosis narrative with the Metrics over time chart: the chart shows the trend, the diagnosis explains why.
  • If a summary says there wasn’t enough data, run a couple more evaluations and come back — it updates on the next run.