Skip to main content

​
Measuring performance

This guide covers how to measure. Performance budgets are defined by code path name in ROADMAP.md in the repository. Measure against those budgets and do not copy numbers into other documents.

​
Model latency bench (local keys)

Script: scripts/bench-model.ts Usage:
  • source ~/.profile && pnpm tsx scripts/bench-model.ts --runs 10
  • Optional environment variables: MINIMAX_API_KEY, MINIMAX_BASE_URL, MINIMAX_MODEL, ANTHROPIC_API_KEY
  • Default prompt: “Reply with a single word: ok. No punctuation or extra text.”
The bench needs real provider keys, so it is not run in CI. Latency depends on the network and the provider, so compare runs from the same host and time window.

​
CLI startup bench

Script: scripts/bench-cli-startup.ts Usage:
  • pnpm tsx scripts/bench-cli-startup.ts
  • pnpm tsx scripts/bench-cli-startup.ts --runs 12
  • pnpm tsx scripts/bench-cli-startup.ts --entry dist/entry.js --timeout-ms 45000
It benchmarks these commands:
  • --version
  • --help
  • health --json
  • status --json
  • status
Output includes the average, p50, p95, min and max, and the exit-code and signal distribution for each command. Run pnpm build first when you benchmark dist/entry.js.