Task Results

Each task owns details/<state>/<readable-task-id>--<sha256>/, where <state> is normal, error or fatal according to the most severe final issue of any attempt; warning-only results are normal, and unfinished tasks stay in running. The hash is computed from the exact task ID; sanitization and truncation affect only its readable prefix.

Files

Path relative to the task directory	Purpose
`task.json`	Shared task identity, category, ground truth, attempt plan, and logical attempt numbers mapped to directory names such as `"1": "attempt-1"`.
`attempt-<n>/result.json`	Complete result of one attempt, including its trajectory, artifact references, and `retry_count`.
`attempt-<n>/checkpoint.json`	Evaluation context and transient scheduler/download state.
`attempt-<n>/artifacts/`	The attempt’s actual collected files. `destination` is relative to this directory.
`attempt-<n>/retries/`	Discarded retry diagnostics and archived outputs; these do not count as independent observations.

Issues are recorded in the result, not in an _error_ filename prefix; the state directory only groups tasks. Reports, reuse, the result browser and offline analysis reconstruct the logical task record by following the index. The example below shows that reconstructed record; on disk the complete attempts payloads live in separate files.

Persisted Task Metadata

The following example shows the task-level task.json after three attempts have been allocated. New directories use attempt-<n>, where n is the logical attempt number starting at 1. During execution, the identity mapping is written first; shared result metadata is published after the attempt payloads are durable.

{
  "schema_version": "agentcompass.task.v3",
  "task_id": "example-task",
  "category": "coding",
  "ground_truth": null,
  "attempt_plan": {
    "k": 3,
    "strategy": "avg"
  },
  "attempts": {
    "1": "attempt-1",
    "2": "attempt-2",
    "3": "attempt-3"
  }
}

There is no task-level result.json. The task retry total and sparse retry_counts map are derived from each attempt’s stored retry_count. An allocated attempt without a result is not a completed observation; a terminal failure without a result payload can retain a retry-only record. Attempt directory names must match their logical numbers: "1": "attempt-1". Cross-run reuse writes attempt-<n> directories in the new run and updates artifact references without changing the source. Retrying an attempt keeps its logical number and archives previous outputs under its own retries/ directory. See Legacy for the previous layout and reuse support.

Task Detail Example

{
  "task_id": "<task-id>",
  "category": "<category>",
  "ground_truth": "<reference-answer>",
  "attempt_plan": {
    "k": 3,
    "strategy": "avg"
  },
  "retry_count": 2,
  "retry_counts": {
    "1": 2
  },
  "attempts": {
    "1": {
      "status": "completed",
      "metrics": {
        "correct": true,
        "reward": 0.82
      },
      "final_answer": "<answer>",
      "trajectory": {},
      "issues": [],
      "artifacts": {},
      "analysis_result": {},
      "meta": {
        "benchmark": {
          "...": "..."
        },
        "harness": {
          "...": "..."
        }
      }
    }
  }
}

Every task detail uses this fixed standard shell. Empty values remain explicit as null, "", or {}. The ... entries only indicate component-specific content; a namespace with no content is written as {}.

Task-Level Fields

Field	Required?	Meaning
`task_id`	Yes	Stable, non-empty task ID supplied by the Benchmark. Leading or trailing whitespace is invalid.
`category`	Yes	Benchmark category used for grouped aggregation, or `null` when none is assigned.
`ground_truth`	Yes	Task-level reference data; it may be `null` for a hidden verifier. It is not repeated inside attempts.
`attempt_plan`	Yes	The `k` and `strategy` used to produce this record.
`retry_count`	Yes	Sum of retries consumed across all logical attempts.
`retry_counts`	Yes	Sparse map from string attempt number to retries consumed by that attempt. Its values sum to `retry_count`.
`attempts`	Yes	Non-empty map keyed by canonical positive integer strings such as `"1"`.

Attempt-Level Fields

Every attempt writes all standard fields. Empty values remain present so task-detail consumers see the same field set for every attempt.

Field	Required?	Meaning
`status`	Yes	`completed`, `skipped`, `run_error`, `eval_error`, `run_error_or_eval_error`, `cancelled`, or `interrupted`. A completed attempt is not necessarily successful.
`metrics`	Yes	Benchmark observations keyed exactly as declared by its Metric Contract. Values are JSON booleans or finite numbers.
`final_answer`	Yes	Text, patch, or structured answer produced by the Model or agent; `null` when unavailable.
`trajectory`	Yes	Harness-normalized interaction trace, or `{}` when unavailable.
`issues`	Yes	Structured unresolved issues; `[]` when empty.
`artifacts`	Yes	Integration-specific artifacts or indexes, or `{}` when empty.
`analysis_result`	Yes	Analyzer output keyed by analyzer family, or `{}` when empty.
`meta`	Yes	Namespaced extension data that is useful but is not a generic Benchmark metric.

`meta` Namespaces

Namespace	Owner and examples
`meta.benchmark`	Benchmark-specific grader diagnostics, component scores, or special fields that are not generic observations.
`meta.harness`	Harness diagnostics and `telemetry`, such as token or latency counters.

Both namespaces are always present and use {} when empty. Their internal component-specific keys are not part of the common task-detail schema. Resolved Environment, Recipe, and network plans are stored in run_info.json.resolved_execution_plans; see Run Records and Diagnostics.

Trajectory Shape

When present, trajectory uses the AgentCompass ACTF_v1.0 shape:

Field	Meaning
`schema_version`	Trajectory schema version.
`steps`	Ordered model, tool, Environment observation, and timing steps.
`started_at`, `finished_at`	Complete trajectory timestamps.

Common step fields include step_id, prompts, assistant content, tool calls, observations, timestamps, and metric token/timing values. Harnesses can omit data they do not produce.

Analysis Results

analysis_result.<analyzer-family> can contain is_badcase, score, details, error, and extra. These are analyzer diagnostics and do not alter attempt.metrics or its status. Run-level analysis files combine analyzer output across attempts separately from Benchmark metric aggregation; see Summary and Analysis Results.

Retry Details

When a failure matches the retry policy and budget remains, AgentCompass writes agentcompass.retry.v1 diagnostics before rerunning the current logical attempt:

{
  "schema_version": "agentcompass.retry.v1",
  "task_id": "<task-id>",
  "attempt": 3,
  "retry": 1,
  "max_retries": 2,
  "stage": "evaluate",
  "scope": "evaluate",
  "matched_pattern": "fatal",
  "issues": [{"severity": "fatal", "phase": "evaluate", "code": "judge_failed", "message": "HTTP 503"}],
  "discarded_result": {}
}

attempt identifies the unchanged logical attempt; retry is its one-based retry number. stage locates the failure, while scope says whether the runtime repeats the complete attempt or only evaluation. discarded_result is diagnostic and is not required to match the strict task-detail shape. Completed sibling attempts remain checkpointed. For example, a retry in attempt 3 does not rerun attempts 1 and 2. The final task detail records the consumed retry counts, while only the terminal attempt payload contributes observations.

Sensitive Content

AgentCompass redacts recognized credential fields before persistence, but answers, trajectories, errors, artifacts, and integration-specific metadata can still contain sensitive task content. Review them before sharing a run directory.

Using AgentCompass

Modules

Other Features

Files

Persisted Task Metadata

Task Detail Example

Task-Level Fields

Attempt-Level Fields

`meta` Namespaces

Trajectory Shape

Analysis Results

Retry Details

Sensitive Content

​Files

​Persisted Task Metadata

​Task Detail Example

​Task-Level Fields

​Attempt-Level Fields

​meta Namespaces

​Trajectory Shape

​Analysis Results

​Retry Details

​Sensitive Content

​Related Pages

Files

Persisted Task Metadata

Task Detail Example

Task-Level Fields

Attempt-Level Fields

`meta` Namespaces

Trajectory Shape

Analysis Results

Retry Details

Sensitive Content

Related Pages