Server facts
mcp-server
Project
- Author
- iris-eval
- Repository
- github.com/iris-eval/mcp-server
- License
- Not specified
- Category
- Cloud & DevOps
Tech
- Runtime / language
- Not specified
- Transports
- stdio
- Remote available
- No
- Authentication
- Not specified
Activity
- GitHub stars
- 9
- Last commit
- Data as of
Tags
- Claude
- Observability
- Tracing
- ai agent
- cost tracking
- evaluation
- llm
- mcp
- monitoring
- pii detection
Overview
Agent debugging in Iris starts with log_trace, which records an execution’s inputs and outputs alongside spans, tool activity, latency, tokens, and cost. Developers can then use evaluate_output for heuristic checks covering completeness, relevance, safety, or spending. Stored executions remain searchable through get_traces, with filters for agents, time windows, and evaluation scores, plus sorting and optional aggregate statistics. Self-hosted storage uses SQLite, while the web dashboard presents trace structure, evaluation outcomes, and cost breakdowns. delete_trace removes an execution and its spans while retaining historical evaluation results.
For checks that need semantic interpretation, evaluate_with_llm_judge calls Anthropic or OpenAI and reports scores, reasoning, dimension breakdowns, and evaluation cost; provider credentials are required. verify_citations focuses on whether referenced sources substantiate claims, combining explicitly enabled source retrieval with LLM assessment. Teams can investigate recurring failures, inspect their custom-rule inventory with list_rules, and use deploy_rule to make new checks persistent. delete_rule retires a custom check without removing earlier evaluation results. These workflows support developers monitoring agent behavior, investigating quality failures, and adapting evaluation policies.
mcp-server setup notes
- mcp-server records stdio only; confirm a network transport upstream before planning remote access.
- mcp-server has no recorded license; confirm reuse terms in its repository.