Use the agent catalog

The agent catalog automatically discovers and tracks your agents. It groups generations by agent name and effective version, then shows usage patterns over time.

Browse agents

Navigate to Agents in the AI Observability plugin to see all discovered agents. Each agent card shows:

Agent name and active model.
Current version hash.
Generation count and error rate.
Last active timestamp.

Use the search bar to filter agents by name. Demo agents seeded through the onboarding wizard display a demo badge.

Understand agent versions

By default, AI Observability computes an agent version as a SHA-256 hash of the system prompt and tool definitions. SDKs can also send an effective version to keep a stable catalog identity when the visible tool surface changes between turns. When the SDK does not send one, changing a prompt or adding, removing, or modifying a tool creates a new version automatically.

Set a stable agent_version in your SDK instrumentation when you want Performance charts to compare declared releases. Without agent_version, the agent can still appear in the catalog, but per-version Performance charts may not have a matching Prometheus metric label.

The version history for each agent shows:

When each version was first and last seen.
Which models each version used.
The tool and prompt footprint for each version.

Compare versions

Use the agent catalog to compare metrics across versions:

Token usage and cost trends.
Error rates and latency distributions.
Evaluation score changes.

This helps you determine whether a prompt change improved or degraded quality.

Analyze prompts

Use the agent detail page to review prompt and tool changes for each version. Prompt insights summarize the active system prompt and highlight patterns that may affect quality, latency, or cost.

When agent rating is configured, run an agent score analysis to get a judge-model assessment of the selected version. The result appears on the agent overview and prompts views so you can compare the rating with quality and performance trends.

Model cards

AI Observability maintains model cards with metadata about each LLM model your agents use. Model cards show pricing tiers, context windows, and capabilities. The catalog syncs model card data periodically to keep information current.