Documentation

Verification

Automated quality checks — build, test, lint — with auto-detection and structured evidence.

Edit on GitHub

Nyzhi can automatically detect and run quality checks for your project — builds, tests, linting, and custom commands. Results are captured as structured evidence with pass/fail status, timing, and output.

How It Works

When the agent runs the verify tool, Nyzhi:

  1. Detects your project type
  2. Selects appropriate checks (or uses your custom ones)
  3. Runs each check sequentially
  4. Returns a structured report with pass/fail evidence

Auto-Detected Checks

Nyzhi detects your project type and selects checks automatically:

Rust Projects

cargo check          # build check
cargo test           # tests
cargo clippy -- -D warnings   # lint

Node.js Projects

npm run build        # build (if script exists)
npm test             # tests
npx eslint .         # lint (if eslint is installed)

Go Projects

go build ./...       # build
go test ./...        # tests
go vet ./...         # lint

Python Projects

python -m pytest     # tests
python -m ruff check .   # lint

Custom Checks

Override auto-detection with explicit checks in your config:

[[agent.verify.checks]]
kind = "build"
command = "make build"

[[agent.verify.checks]]
kind = "test"
command = "pytest -x --tb=short"

[[agent.verify.checks]]
kind = "lint"
command = "flake8 src/"

[[agent.verify.checks]]
kind = "custom"
command = "scripts/integration-test.sh"

Check kinds: build, test, lint, custom.

Verification Reports

The verify tool returns a structured report:

✓ build   cargo check              2.3s
✓ test    cargo test               8.1s
✗ lint    cargo clippy -- -D warnings  1.5s
          → warning: unused variable `x`

Each piece of evidence includes:

  • Status — pass or fail
  • Command — what was run
  • Duration — how long it took
  • Output — stdout/stderr (shown on failure)
  • Timestamp — when it ran

Evidence Freshness

Evidence has timestamps, so you can check freshness:

Evidence::is_fresh(max_age)

The agent uses this to avoid re-running checks that were just verified.

CI Integration

nyz ci-fix

Automatically fix CI failures:

# From a CI log file
nyz ci-fix --log-file ci-output.log --format junit

# From stdin
cat ci.log | nyz ci-fix

# Auto-commit the fix
nyz ci-fix --log-file ci.log --commit

Supported log formats: auto, junit, tap, plain.

Using exec with Verification

nyz exec --full_auto "run verification checks and fix any failures"

Best Practices

  • Run checks before committing — add a pre_tool_use hook for git_commit that runs verify
  • Keep checks deterministic — flaky tests create noise and waste time
  • Keep checks fast — slow checks degrade the agent’s iteration speed
  • Use custom checks for project-specific requirements
  • Include both local commands and CI links in PR descriptions for large changes

Example: Full Verification Setup

# Project config: <project>/.nyzhi/config.toml

[[agent.verify.checks]]
kind = "build"
command = "cargo check --workspace"

[[agent.verify.checks]]
kind = "test"
command = "cargo test --workspace"

[[agent.verify.checks]]
kind = "lint"
command = "cargo clippy --workspace --all-targets -- -D warnings"

[[agent.verify.checks]]
kind = "custom"
command = "cargo fmt --all --check"

# Auto-verify after each turn
[[agent.hooks]]
event = "after_turn"
command = "nyz verify"
timeout = 120

Next Steps

Back to overview