Skip to content
AgentV

Quick Start

Follow these steps to create and run your first evaluation.

1. Install AgentV plugin

Section titled “1. Install AgentV plugin”
Terminal window
npx allagents plugin marketplace add EntityProcess/agentv
npx allagents plugin install agentv-dev@agentv

2. Ask Claude to bootstrap AgentV in this repo

Section titled “2. Ask Claude to bootstrap AgentV in this repo”
Set up AgentV in this repo.

The onboarding skill ensures CLI/setup prerequisites and runs:

Terminal window
agentv init

3. Configure environment variables

Section titled “3. Configure environment variables”

The init command creates a .env.example file in your project root.

  1. Copy .env.example to .env
  2. Fill in your API keys, endpoints, and other configuration values
  3. Update the environment variable names in .agentv/targets.yaml to match those defined in your .env file

4. Create an eval

Section titled “4. Create an eval”

Create ./evals/example.yaml:

description: Math problem solving evaluation
execution:
  target: default


tests:
  - id: addition
    criteria: Correctly calculates 15 + 27 = 42


    input: What is 15 + 27?


    expected_output: "42"


    assert:
      - name: math_check
        type: code-judge
        command: [./validators/check_math.py]

5. Run the eval

Section titled “5. Run the eval”
Terminal window
agentv eval ./evals/example.yaml

Results appear in .agentv/results/eval_<timestamp>.jsonl with scores, reasoning, and execution traces.

Next Steps

Section titled “Next Steps”