mcp-server

Iris Eval MCP Server

Iris helps AI agent developers inspect executions, assess response quality, and track spending through MCP tools. It combines stored traces, configurable evaluation rules, LLM judging, and citation checks with a web dashboard.

Server facts

mcp-server

Project

Author
iris-eval
License
Not specified

Tech

Runtime / language
Not specified
Transports
stdio
Remote available
No
Authentication
Not specified

Activity

GitHub stars
9
Last commit
Data as of

Tags

  • Claude
  • Observability
  • Tracing
  • ai agent
  • cost tracking
  • evaluation
  • llm
  • mcp
  • monitoring
  • pii detection

Overview

Agent debugging in Iris starts with log_trace, which records an execution’s inputs and outputs alongside spans, tool activity, latency, tokens, and cost. Developers can then use evaluate_output for heuristic checks covering completeness, relevance, safety, or spending. Stored executions remain searchable through get_traces, with filters for agents, time windows, and evaluation scores, plus sorting and optional aggregate statistics. Self-hosted storage uses SQLite, while the web dashboard presents trace structure, evaluation outcomes, and cost breakdowns. delete_trace removes an execution and its spans while retaining historical evaluation results.

For checks that need semantic interpretation, evaluate_with_llm_judge calls Anthropic or OpenAI and reports scores, reasoning, dimension breakdowns, and evaluation cost; provider credentials are required. verify_citations focuses on whether referenced sources substantiate claims, combining explicitly enabled source retrieval with LLM assessment. Teams can investigate recurring failures, inspect their custom-rule inventory with list_rules, and use deploy_rule to make new checks persistent. delete_rule retires a custom check without removing earlier evaluation results. These workflows support developers monitoring agent behavior, investigating quality failures, and adapting evaluation policies.

mcp-server setup notes

  • mcp-server records stdio only; confirm a network transport upstream before planning remote access.
  • mcp-server has no recorded license; confirm reuse terms in its repository.

Related MCP servers