Environment Resource Limits

Environment resource parameters limit what a local instance can use or specify the CPU, memory, storage, and GPU requested for a remote instance. Use them to keep one task from consuming excessive resources or to request a remote instance that matches Benchmark requirements. They do not limit the AgentCompass host process, model service, or other external services.

host_process runs commands directly on the host and cannot enforce Environment-level CPU, memory, storage, or GPU limits. Choose another provider when you need resource isolation.

Separate Resources from Scheduling

These four settings solve different problems:

Setting	What it controls
Environment resource parameters	Resources available to or requested for one Environment instance. Set them as Environment parameters; on the CLI, use `--env-params`.
`--task-concurrency`	The maximum number of Benchmark tasks processed at the same time in one evaluation.
`--provider-limit <provider>=<count>`	The maximum concurrent task attempts handled by one provider in the current AgentCompass process.
`--env-open-qps <provider>=<qps>`	How many Environment creations can begin per second for one provider. It limits creation rate, not the number of running instances.

For example, Docker resources.cpu: 2 limits each container to two cores, while --task-concurrency 8 permits up to eight tasks to be processed concurrently. Neither setting replaces the other. Task and verifier Environments are also separate resource allocations. When a Benchmark requires a fresh verifier Environment, AgentCompass normally closes the task Environment before creating it. They overlap only when --keep-environment retains the task Environment. See Run Controls for the full behavior of concurrency, open rate, and --keep-environment.

Provider Capabilities and Units

AgentCompass exposes one provider-neutral resource model. Provider adapters convert these values into their platform-native fields and units.

Field	Common format
`cpu`	Positive numeric core count. A provider may require a whole number.
`memory_mb`	Positive integer memory size in MiB. Do not add a unit suffix.
`storage_mb`	Positive integer storage size in MiB. Do not add a unit suffix.
`gpu`	Non-negative integer GPU count. An explicit `0` clears an inherited `gpu_type`.
`gpu_type`	Non-empty GPU model string. Setting it without `gpu` implies `gpu: 1`.
`ignore_gpu_type`	Boolean override directive. Set it to `true` to remove an inherited `gpu_type` constraint while preserving the GPU count.

Providers support subsets of that model:

Provider	`cpu`	`memory_mb`	`storage_mb`	`gpu`	`gpu_type`
`host_process`	No	No	No	No	No
`docker`	Yes	Yes	Yes	Yes	No
`daytona`	Yes, integer	Yes	Yes	Yes, at most 1	Yes
`modal`	Yes	Yes	No	Yes	Yes
`opensandbox`	Yes	Yes	No	Yes	No

AgentCompass reports an unsupported requested field before opening an Environment. storage_mb is the exception: a provider that cannot enforce it ignores it with a warning, so task-declared disk requirements do not prevent otherwise compatible runs. GPU model constraints remain fail-closed by default. If a Benchmark requires a specific model but you intentionally want the selected provider to allocate any available GPU, set ignore_gpu_type: true explicitly:

agentcompass run <benchmark> <harness> "$MODEL_NAME" \
  --env <provider> \
  --env-params '{"resources":{"gpu":1,"ignore_gpu_type":true}}'

This removes only gpu_type; it does not remove or change gpu. AgentCompass logs a warning when it applies the directive. The directive follows the same precedence and phase scoping as other resource fields, so it can also be placed in run_resources or evaluation_resources. A higher-precedence gpu_type restores strict model selection for that phase. Do not use this override when the task depends on a particular GPU architecture, memory capacity, or performance profile, and record it when comparing results. Use the following command to inspect the exact fields and defaults accepted by the installed revision:

agentcompass config docs env <provider>

Each provider page explains its field formats, account quotas, and operating requirements in more detail.

Configure Resources

Scope Resources for Fresh Evaluation

Benchmarks with evaluation_environment_mode="fresh" create separate run and evaluation Environments. Use these fields inside --env-params to control their resources:

Field	Scope
`resources`	Common override applied to both the run and fresh evaluation Environment. For other modes, it applies to the single shared Environment.
`run_resources`	Override applied only to the task run Environment.
`evaluation_resources`	Override applied only to a fresh evaluation Environment. AgentCompass rejects this field when the effective evaluation mode is not `fresh`.

Phase-specific fields take precedence over the same fields in resources. For example, this configuration gives both Environments 8192 MiB of memory, while assigning 4 CPUs to the run Environment and 2 CPUs to the evaluation Environment:

agentcompass run swebench_verified mini_swe_agent "$MODEL_NAME" \
  --env docker \
  --benchmark-params '{"sample_ids":["astropy__astropy-12907"]}' \
  --env-params '{
    "resources":{"memory_mb":8192},
    "run_resources":{"cpu":4},
    "evaluation_resources":{"cpu":2}
  }'

Benchmark task metadata remains the lowest-precedence resource source. A common explicit resources value overrides the corresponding task fields for both fresh Environments; run_resources and evaluation_resources then override the corresponding common fields. When a fresh Benchmark does not define separate evaluation resources, the evaluation Environment falls back to its common task resources. The following examples use the same unified resource shape with Docker, Daytona, and Modal. Each one selects a single task through sample_ids and assigns 2 CPU cores and 6144 MiB of memory to each Environment. These values demonstrate the syntax; they are not a recommended Benchmark configuration. The examples use agentcompass run. See Configure an Environment for configuration-file, Python SDK, and launch orchestration-file forms.

Docker

agentcompass run swebench_verified mini_swe_agent "$MODEL_NAME" \
  --env docker \
  --benchmark-params '{"sample_ids":["astropy__astropy-12907"]}' \
  --env-params '{"resources":{"cpu":2,"memory_mb":6144}}'

The Docker adapter translates these values into Docker CPU and memory arguments. See Docker resource parameters for provider requirements.

Daytona

agentcompass run swebench_verified mini_swe_agent "$MODEL_NAME" \
  --env daytona \
  --benchmark-params '{"sample_ids":["astropy__astropy-12907"]}' \
  --env-params '{"resources":{"cpu":2,"memory_mb":6144}}'

The recipe for this combination selects the task image, so the request applies to an image-based sandbox. See Daytona resource parameters for provider requirements.

agentcompass run swebench_verified mini_swe_agent "$MODEL_NAME" \
  --env modal \
  --benchmark-params '{"sample_ids":["astropy__astropy-12907"]}' \
  --env-params '{"resources":{"cpu":2,"memory_mb":6144}}'

The Modal adapter passes the MiB memory value to Modal. See Modal resource parameters for provider requirements.

Docker applies storage_mb through --storage-opt size=.... If the Docker daemon reports that its storage driver cannot enforce this option, AgentCompass logs a warning and starts the container without a storage limit. A remote provider may also reject a request because of account quota, regional capacity, or an unavailable instance shape.

Recipe Resources and Explicit Overrides

Some recipes read task resource requirements from a Benchmark and place them in the unified resource model. The selected provider adapter then performs platform-specific validation and conversion. Built-in recipes usually preserve compatible explicit resource values, but the exact adaptation still depends on the recipe and Benchmark. As a result:

to reproduce the Benchmark resource conditions, start with the defaults supplied by its recipe; and
to compare another resource profile, override it explicitly and record the change with the results.

Resource changes can affect task completion and scores. Do not combine runs made under different resource limits as though they used the same evaluation conditions.

Estimate Aggregate Capacity

Estimate capacity in this order:

Start with the resource requirements supplied by the Benchmark or recipe.
Run one representative task and observe peak memory, CPU use, disk growth, and verifier needs.
Leave headroom for dependency installation, compilation, and caches.
Estimate aggregate use from per-instance resources and actual concurrency, then adjust task and provider limits.
Increase concurrency gradually, reducing it when OOM failures, creation errors, or sustained queueing appear.

Capacity planning should focus on how many Environment instances can exist at the same time. --env-open-qps changes only how quickly new instances begin creation; it does not limit the number of running instances.

Troubleshoot Resource Problems

Symptom	Common cause	What to do
Docker reports exit code `137`, `OOMKilled`, or an abrupt process exit	The container exceeded its memory limit.	Inspect container state and peak memory, then compare the limit with the Benchmark requirement.
The host becomes unresponsive with several tasks running	Aggregate Environment demand exceeds host capacity.	Lower `--task-concurrency` or the relevant `--provider-limit`.
Daytona or Modal rejects instance creation	The field format, requested shape, regional capacity, or account quota is invalid.	Check the provider page and account console, then validate the configuration with one instance.
CPU use is low but the task still times out	Time is spent waiting for the model, network, or Harness rather than for CPU.	Inspect phase logs before increasing CPU.
The Docker writable layer fills	Task output exceeds its capacity, or the storage driver cannot enforce the configured limit.	Check the storage driver and use a supported storage option or a more suitable image layout.
A GPU is not visible inside the Environment	The host runtime, image, driver, or provider GPU request is incompatible.	Validate the provider’s GPU configuration independently before running the evaluation.

Using AgentCompass

Modules

Other Features

Separate Resources from Scheduling

Provider Capabilities and Units

Configure Resources

Scope Resources for Fresh Evaluation

Docker

Daytona

Recipe Resources and Explicit Overrides

Estimate Aggregate Capacity

Troubleshoot Resource Problems

​Separate Resources from Scheduling

​Provider Capabilities and Units

​Configure Resources

​Scope Resources for Fresh Evaluation

​Docker

​Daytona

​Modal

​Recipe Resources and Explicit Overrides

​Estimate Aggregate Capacity

​Troubleshoot Resource Problems

​Related Pages

Separate Resources from Scheduling

Provider Capabilities and Units

Configure Resources

Scope Resources for Fresh Evaluation

Docker

Daytona

Modal

Recipe Resources and Explicit Overrides

Estimate Aggregate Capacity

Troubleshoot Resource Problems

Related Pages