Choosing a Model (OpenAI)

Testronaut supports multiple OpenAI models for agentic (tool-using) workflows.
You can pick a default during testronaut --init, store it in testronaut-config.json, and override it per run via the CLI.

Model availability and rate limits depend on your OpenAI account and project. Testronaut uses conservative startup limits and can learn effective limits from API response headers when provided.

---

Supported model families

Current models shown first during initialization:

• GPT-5.6 Sol (gpt-5.6) — Default; flagship capability
• GPT-5.6 Terra (gpt-5.6-terra) — Balanced intelligence and cost
• GPT-5.6 Luna (gpt-5.6-luna) — Cost-sensitive, high-volume workloads
• GPT-5.5 (gpt-5.5) — Previous flagship model

The legacy picker group includes gpt-5.4, gpt-5.4-mini, gpt-5.4-nano, gpt-5.2, and the previously supported GPT-5, GPT-4, o3, and o4-mini aliases. There is no stable general-purpose GPT-5.3 alias, so Testronaut does not list one.

Testronaut currently uses Chat Completions. For GPT-5.6 function-tool calls it sets reasoning_effort to none, because reasoning plus function tools requires the Responses API. A future Responses migration will enable reasoning and tool calling together.

You can always add more choices as OpenAI releases them—Testronaut reads the model id you select and forwards it directly to the API.

---

How Testronaut decides which model to use

1. CLI override (highest precedence)
   If you pass --model <id> on the command line, that model is used for the entire run.

2. Environment override
   If TESTRONAUT_MODEL is set, it overrides the config file.

3. Project config
   testronaut-config.json stores { "provider": "openai", "model": "<id>" } after init.

4. Fallback
   If none of the above are present, Testronaut defaults to gpt-5.6.

The chosen model is surfaced in:

• The CLI JSON and HTML reports (results.llm.provider & results.llm.model)
• The companion app (header and table pills)

---

Set the default model (config)

After running testronaut --init, you’ll have testronaut-config.json.
Edit it to set the provider/model:

testronaut-config.json

{
  "initialized": true,
  "provider": "openai",
  "model": "gpt-5.6",
  "outputDir": "missions/mission_reports",
  "projectName": "your-project",
  "maxTurns": 20
}

Examples: gpt-5.6, gpt-5.6-terra, gpt-5.6-luna, gpt-5.5, and the legacy aliases listed above.

Override per run (CLI) Use --model to override for a single run:

Run all missions with GPT-5.6 Terra

testronaut --model gpt-5.6-terra

Run a specific mission with o3

testronaut downloadDocument.mission.js --model o3

This sets the model for the whole execution and shows up in the generated reports.

Override via environment

macOS/Linux

export TESTRONAUT_MODEL=gpt-5.6-luna
testronaut

Windows (PowerShell)

$env:TESTRONAUT_MODEL="gpt-5.6-luna"
testronaut

---

Cost / benefit guide (quick picks)

Exact pricing and limits change; treat this as a practical guide. Choose based on the mission’s needs (speed vs. depth vs. cost).

Family/Model	Strengths	Typical Use	Trade-offs
GPT-5.6 Sol	Flagship reasoning and tool use	Complex agentic missions	Highest-cost GPT-5.6 tier
GPT-5.6 Terra	Balance of intelligence and cost	General testing	Less peak capability than Sol
GPT-5.6 Luna	Lowest-cost GPT-5.6 tier	Smoke tests and high-volume runs	Less peak capability than Terra/Sol
GPT-5.5	Previous flagship	Compatibility and comparison runs	Legacy generation

---

Recommendations

Default / best general agent: gpt-5.6

Balanced: gpt-5.6-terra

High-volume / cost-sensitive: gpt-5.6-luna

---

Rate limits & dynamic backoff (FYI)

Testronaut ships with token-aware throttling:

It estimates tokens per turn, warns when a request approaches a known context window, and backs off when nearing per-minute token limits. Context warnings do not truncate requests.

If a 429 occurs, it can learn updated limits from response headers (when provided).

New/unsupported tokenizers are handled with sensible fallbacks—so you can safely pick future model IDs.

If you select a model that your account can’t use, expect quota or capability errors; choose another model available to your OpenAI project.

---

Troubleshooting

“Invalid model: xyz”
Ensure the model id is spelled exactly as the API expects.
If it’s a new family (e.g., GPT-5) and your tokenizer library doesn’t recognize it, Testronaut still estimates tokens via fallbacks—runs will continue.

“429: rate limit exceeded”
Pick a high-volume model such as gpt-5.6-luna, or reduce concurrency. Try again later or contact OpenAI about quota.
Consider setting TESTRONAUT_MODEL to a lighter model for large suites.

Examples Set default once, override ad-hoc:

Configure the default model in testronaut-config.json

testronaut --init

Temporary override to o3 for a tricky mission

testronaut checkoutFlow.mission.js --model o3

Project-wide switch via ENV (CI friendly):

TESTRONAUT_MODEL=gpt-4.1-mini testronaut

---

Reports

The JSON & HTML reports show llm.provider and llm.model.

The companion app displays these as header text and colorful pills in the table.