> ## Documentation Index
> Fetch the complete documentation index at: https://opencompass-docs-preview-pr-335-0.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Configure a Model

Configure a model id, endpoint, credential, API protocol, and endpoint-specific inference parameters.

AgentCompass does not maintain a fixed model-name registry. The model id comes from the endpoint you evaluate and is the
third positional argument to `agentcompass run`:

```bash theme={"system"}
agentcompass run <benchmark> <harness> "$MODEL_NAME"
```

The runtime stores that id together with its endpoint, credential, API protocol, and inference parameters in one
`ModelSpec`. The selected harness decides how to consume the spec.

## Model API Protocol List

Model ids vary by provider, but AgentCompass defines three protocol ids. They are also included in the output written by
`agentcompass list dump`.

| id | description |
| - | - |
| [`openai-chat`](/en/user_guide/modules/models/openai_chat) | OpenAI-compatible Chat Completions protocol for `/v1/chat/completions` style endpoints. |
| [`openai-responses`](/en/user_guide/modules/models/openai_responses) | OpenAI Responses API protocol for `/v1/responses` style endpoints and Responses-specific reasoning or stateful tool calls. |
| [`anthropic`](/en/user_guide/modules/models/anthropic_messages) | Anthropic Messages protocol for Claude-style `/v1/messages` endpoints. |

Protocol support also depends on the selected harness. An endpoint implementing OpenAI Chat does not make it compatible
with a harness that requires Responses or Anthropic Messages behavior.

`--model-api-protocol` accepts any protocol id in the table. When the option is omitted or set to `auto`, the selected
harness chooses the protocol; for example, `codex` defaults to `openai-responses`, while `claude_code` uses `anthropic`.

The option also accepts an ordered JSON array such as `'["openai-responses","openai-chat"]'`. The harness selects the
first protocol it supports. The array is not a request-failure fallback and cannot contain `auto`.

## Configure the Model Spec

The [Run Parameter Reference](/en/user_guide/using_agentcompass/cli/run#parameter-reference) introduces the model
positional argument and `--model-*` flags. Together they construct these `ModelSpec` fields:

| ModelSpec field | CLI input | Type and default | What it controls |
| - | - | - | - |
| `id` | Primary `MODEL` positional argument | Required string | Model name sent to the endpoint and model-name segment used in result paths. |
| `base_url` | `--model-base-url <url>` | String, default `""` | API base URL. It may be empty when the selected client resolves a provider default. |
| `api_key` | `--model-api-key <key>` | String, default `""` | Endpoint credential. Pass an environment-variable reference instead of a literal secret. |
| `api_protocol` | `--model-api-protocol <protocol>` | String or ordered string list, unspecified by default | Chooses how the harness communicates with the endpoint. |
| `params` | `--model-params <json>` | JSON object, default `{}` | Carries inference, client reliability, reasoning, and provider-specific request fields. |

### Configure Connection Details

One `agentcompass run` command contains one `ModelSpec`. To compare multiple model ids, declare one named request per
model with [`agentcompass launch`](/en/user_guide/using_agentcompass/cli/launch); this makes endpoint and inference-setting
differences explicit instead of copying one implicit comparison template.

Export model connection values once and keep credentials out of command history:

```bash theme={"system"}
export MODEL_NAME=""
export MODEL_BASE_URL=""
export MODEL_API_KEY=""

agentcompass run <benchmark> <harness> "$MODEL_NAME" \
  --model-base-url "$MODEL_BASE_URL" \
  --model-api-key "$MODEL_API_KEY" \
  --model-api-protocol openai-chat
```

### Configure the `params` Field

The `--model-params <json>` object does not have one AgentCompass-wide generation schema. Its accepted fields are the
intersection of three contracts:

```text theme={"system"}
model params
  ├─ fields consumed or translated by the selected harness
  ├─ request fields defined by the selected API protocol
  └─ fields supported by the endpoint and model deployment
```

```bash theme={"system"}
agentcompass run <benchmark> <harness> "$MODEL_NAME" \
  --model-params '{
    "temperature": 0,
    "max_tokens": 4096
  }'
```

Common model parameters fall into the following categories:

| Field family | Examples | How to choose it |
| - | - | - |
| Sampling | `temperature`, `top_p`, seed fields | Match the official benchmark setting. For debugging, use deterministic values only when the endpoint supports them. |
| Output budget | `max_tokens`, `max_output_tokens` | Use the field required by the protocol and keep it below the endpoint context budget while avoiding valid-output truncation. |
| Reasoning | `reasoning_effort`, thinking controls | Use the exact model/provider request shape and record it in alignment reports. Some providers require nested fields. |
| Client reliability | `timeout`, `max_retries` | Bound one model request and its transport retries. Keep these separate from harness timeout and AgentCompass task retries. |
| Provider extensions | `extra_body` and provider-defined fields | Pass only documented endpoint options. A field accepted by one OpenAI-compatible deployment may be rejected by another. |

These names are not portable by default. For example, OpenAI Chat commonly uses `max_tokens`, OpenAI Responses uses
Responses-specific request fields such as `max_output_tokens`, and Anthropic Messages has its own required request
shape. The protocol pages document how AgentCompass forwards each request.

The selected harness page is also authoritative. A CLI-based harness may translate `ModelSpec` into its own config file
instead of forwarding `--model-params` directly through AgentCompass's native protocol clients.

`--model-params` must be valid JSON. CLI values deep-merge over matching keys in `model.params` from configuration files.
Pass only the fields that differ from the effective endpoint and harness defaults.

## Configure Judge and Analysis Models

Some benchmarks and analyzers use an additional model spec for judging or qualitative analysis. These nested specs use
the same concepts—model id, base URL, API key, protocol, and params—but belong to their owning component:

* Benchmark judge models are normally under `--benchmark-params`, for example `judge_model`.
* Analyzer models are under `--analysis-params`, such as `QualitativeAnalyzer`.
* Tool-specific summarization models may belong to the selected harness.

Do not put judge credentials into the primary `--model-params` object unless the owning benchmark or harness explicitly
documents that schema.

## Diagnose Model Configuration

| Symptom | Likely cause |
| - | - |
| Provider not detected | The underlying client expects a provider-qualified model id, or the selected harness uses a different protocol. |
| 401 or authentication error | The API key is missing, expired, or belongs to another endpoint. |
| 404 or model not found | The model id is not served by the configured base URL. |
| 429 or rate-limit error | Task concurrency, harness parallelism, requests per minute, or token throughput exceeds endpoint capacity. |
| Unsupported protocol | The selected harness does not support the requested API protocol. |
| Invalid request field | A parameter was passed using the wrong protocol or provider-specific shape. |
| Repeated truncation | The output limit is too small or the total context budget is exhausted. |

Reduce failures to one task and inspect the harness and endpoint error before changing multiple model parameters at once.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.