How Vettd Scoring Works

Transparent methodology for trust and risk analysis

What We Scan

Vettd analyzes AI agents, tools, and integrations by examining their configuration files, dependencies, and runtime behavior. The scanner (Proov) inspects:

  • Prompt configuration files — .cursorrules, agents.md, *.prompt.md, copilot-instructions.md
  • MCP server configurations — mcp.json, transport types, network exposure, credential handling
  • Container configurations — Dockerfiles, docker-compose files, exposed ports, base images
  • Agent declarations — capability mapping, tool references, execution model
  • Agentic app orchestrations — multi-agent workflows, service dependencies
  • Network evidence — API endpoints, environment variables, firewall rules

Risk Scoring (0–100)

Every artifact receives a risk score from 0 (minimal risk) to 100 (critical concern). Scores are calculated from 13+ signal types:

  • Shell access detected → higher risk
  • Credential references in configuration → higher risk
  • Network exposure (public endpoints) → higher risk
  • API endpoint extraction → moderate risk
  • Browser access capabilities → moderate risk
  • File system access → moderate risk
  • Execution tokens present → variable risk
  • Container privilege levels → variable risk

Scores combine additively — an artifact with shell access AND exposed credentials scores higher than either alone.

Trust Scores

Trust scores invert the risk: trust = 100 - risk. A trust score of 85 means only 15 points of risk signals were detected.

For agents specifically, trust scores include a breakdown showing what helped and what hurt:

Shell execution detected: -20

No credential exposure: +0 (neutral)

Badges

Three badge levels based on overall analysis:

VettdPassed all verification checks. Low risk, transparent configuration, no dangerous patterns detected.
ConditionalPassed with conditions. Some risk signals present but within acceptable thresholds. May need manual review for specific use cases.
FlaggedFailed one or more critical checks. High risk signals detected — credential exposure, unrestricted shell access, or dangerous patterns.

What We Check Today

We believe in being honest about scope. Here is what Vettd currently checks:

  • ✅ Configuration file analysis (prompts, MCP, containers, agent declarations)
  • ✅ Credential and secret pattern detection
  • ✅ Network exposure and transport analysis
  • ✅ Capability mapping (shell, filesystem, browser, network, database access)
  • ✅ Custom detection rules (extensible TOML rule engine)
  • ✅ Risk scoring with 13+ signal types

What we're building next:

  • 🔜 Framework-specific detection (LangGraph, CrewAI, AutoGen)
  • 🔜 Deep tool introspection (beyond name detection)
  • 🔜 Cross-platform firewall analysis
  • 🔜 Continuous monitoring and re-scanning

Our Principles

  • Scores are informational, not guarantees
  • We never access or store prompt content (privacy-first)
  • Our methodology is public and will evolve
  • If you disagree with a score, contact us: hello@agentichighway.ai