v0.8: chaos workflows, declarative experiments and explosion radius

Available in 0.8.0. The core installation stays standalone. YAML experiment files need the optional extra (pip install 'agentic-chaos[yaml]'). JSON and TOML experiment files need nothing extra.

The offline examples need no provider and no network:

examples/chaos_workflow_demo.py uses the Python API.
examples/workflows/ has a support agent and two experiment files.
The demo GIF is generated from their real output.

Explosion radius: `Scope`

A Scope limits where a fault is allowed to fire. Every built-in fault, and cascade(), accepts scope=:

from agentic_chaos import RateLimitStormFault, Scope

RateLimitStormFault(
    burst_count=3,
    scope=Scope(
        steps=["retriever", "planner"],  # only these steps
        providers=["openai"],            # only OpenAI calls
        exclude_steps=["safety_check"],  # never this one
    ),
)

Steps are matched against the step_name of a call. wrap_tool() and wrap_node() use the tool or node name as step_name.
Providers are matched against chaos_provider. patch_openai, patch_anthropic and patch_google set it for you. For direct calls, pass it yourself: chaos_call(fn, ..., chaos_provider="openai"). It is never forwarded to fn.
Exclusions always win. exclude_steps and exclude_providers override any inclusion.
Include lists fail closed. An unlabelled call is outside any scope that names steps or providers.
Scope is checked before probability sampling. Out-of-scope calls don't consume random draws, burst counts or dry-run preview copies, and they don't appear in dry-run plans. A scoped run with a seed therefore draws exactly what an unscoped run would draw over only the in-scope calls.
Cascades check their own scope and the scope of the pending stage.
Profiles accept scopes too: add a [profiles.NAME.parameters.FAULT.scope] table with the same four keys.

Custom faults that call BaseFault.__init__ get scoping automatically. Others can set a scope attribute or override in_scope().

Chaos workflows

A workflow is a multi-stage campaign:

from agentic_chaos import RateLimitStormFault, Scope, TokenTimeoutFault, cascade
from agentic_chaos.workflows import (
    ChaosWorkflow, HealthCheck, Parallel, Step, Suspend, Template,
)

storm = Template("storm", lambda burst=2: [
    Step("rate-limits", faults=[RateLimitStormFault(burst_count=burst)]),
    HealthCheck(fn=my_agent, name="answers", expect_contains="Answer:"),
])

workflow = ChaosWorkflow("resilience-suite", steps=[
    HealthCheck(fn=my_agent, query="Are you working?"),
    storm.use(name="warm-up"),
    Parallel(Step("tools", faults=["tool_failure"]),
             Step("memory", faults=["memory_corruption"])),
    Suspend("approve-escalation", reason="the next stages are louder"),
    storm.use(name="escalate", burst=5),
    Step("full-blast", faults=[cascade(first=RateLimitStormFault(burst_count=2),
                                       then=TokenTimeoutFault(hang_seconds=1))]),
], scope=Scope(exclude_steps=["safety_check"]))

report = workflow.run(my_app, approve={"approve-escalation"})

Node types

Node	Behavior
`Step(name, faults, *, target, repeat, contracts, breaker, dry_run, scope)`	Runs the target `repeat` times inside a fresh `chaos_session`. Fails when the target raises (or a script exits non-zero) or a contract doesn't pass. Reports `aborted` when its breaker trips.
`HealthCheck(fn, query, *, name, expect_success, expect_contains, expect, retries, retry_delay_seconds, halt_on_failure)`	A steady-state gate that runs with no chaos session active. With `expect_success=False` it passes only when the call raises. A failed check halts the workflow unless `halt_on_failure=False`.
`Parallel(*branches, name, max_workers)`	Runs each branch (a node or a list of nodes) in its own thread and its own empty context, so each branch has its own chaos session and topology tracker. Waits for every branch before the next node.
`Suspend(name, reason, approve)`	A human-in-the-loop gate. The approver returns `True` to continue, `False` to reject, or `None` (no decision yet) to stop the run as `suspended`.
`Template(name, build, defaults)` / `.use(name, params)`	A reusable node sequence. `build` is a callable, or a fixed list of nodes. Each use prefixes node names with its own name (`escalate/rate-limits`).

Semantics

Fresh fault instances. Faults are deep-copied every time a step runs. Running a workflow twice, or using a template twice, never shares burst counts, decay progress or random streams. Fault names are resolved to fresh instances.
Stopping. By default, the first failing step stops the workflow. stop_on_failure=False keeps going, but the final status is still failed. Remaining nodes are reported as skipped, so a report always shows the full plan.
Workflow-level scope is intersected with every fault's scope, together with each step's own scope.
Workflow-level breaker is the default for steps without their own. Breaker state is per step, because each step is its own session.
Approving gates. approve= takes a callable (gate, report_so_far) -> bool | None, a set of gate names, or True/False. A gate's own approve callable takes precedence.
Resuming. resume_from="gate" skips every node up to and including that top-level gate, which is treated as approved. Skipped nodes are reported as skipped and the gate as resumed.
Dry run. dry_run=True previews every step. Targets and health checks still execute, as with all Chaos previews. Suspends are reported as would_suspend and don't stop the run.
Not allowed: running a workflow inside an active chaos_session(), and putting a Suspend inside a Parallel block.
Names. Node names must be unique. Unnamed health checks and parallel blocks are numbered (health-check-1, parallel-1, ...).

Statuses

Level	Statuses
Node	`passed`, `failed`, `aborted`, `skipped`, `approved`, `rejected`, `suspended`, `resumed`, `would_suspend`
Workflow	`passed`, `failed`, `aborted`, `halted` (a health check failed), `rejected`, `suspended`

Reports

WorkflowReport (kind: chaos_workflow, schema_version: "1.0") holds:

the result of every node, including parallel branches;
each step's native ChaosReport and contract results;
every step's events in a top-level chaos_events list.

Each event gains detail.workflow = {id, name, step}. Because events sit at the top level, a workflow report file loads wherever a ChaosReport or an AgenticLens Workflow is read.

report.to_chaos_report() merges every step into one native report:

events, plans, cost, injections and in-flight counts;
call records, renumbered globally, each with a workflow_step;
the first escaped error, and the first abort;
topologies, unioned by node name;
a workflow summary, a new optional ChaosReport field.

Use the merged report with check_contracts() and to_aios_run(). The AIOS export carries the workflow summary in the producer extension (io.deepagentlabs.agentic-chaos). There is no core AIOS schema change.

Declarative experiments

Experiments separate the definition of a chaos test from its execution, so SREs and QA can write, review and version them next to the code.

Single-stage experiment

# examples/workflows/rate-limit-recovery.yaml
name: rate-limit-recovery
target: support_agent.py:handle # resolved relative to this file
faults:
  - type: rate_limit_storm
    burst_count: 3
    retry_after: 0
    scope: { steps: [llm], providers: [openai] }
assertions:
  - completes_within: 30
  - max_retries: { n: 5, step: llm }
  - recovered_after_failure: true
  - no_unhandled_error
safety: { max_injections: 20 }
schedule: { cron: "0 2 * * *", timezone: UTC }

Multi-stage workflow

Replace faults/assertions with a workflow: list of one-key nodes: step, health_check, parallel, suspend and use. Reusable templates take params defaults, and use: {template, name, with} overrides them. See examples/workflows/resilience-suite.yaml.

In a template, a value that is exactly "${name}" is replaced by the parameter and keeps its type, so burst_count: "${burst}" stays an integer. Inside a longer string, the parameter is interpolated as text. Write $${ to get a literal ${.

Top-level fields

Field	Meaning
`version`	Optional; `1`
`name`	Required
`description`	Optional text
`target`	`app.py` (a script run as `__main__`), `app.py:function` or `package.module:function`. Relative paths resolve against the experiment file. A file target's directory is put on `sys.path`, so the file can import modules next to it.
`faults`	Single-stage form only. A fault is a name, or a mapping with `type`, its constructor parameters and an optional `scope`. `type: cascade` takes `first`, `then` and `delay_between`.
`assertions`	Single-stage form only. Each item is a contract name, `{contract: value}` (the first parameter) or `{contract: {param: value}}`.
`repeat`	Single-stage form only. How many times to run the target.
`workflow`	The multi-stage form, a list of nodes.
`templates`	Reusable node sequences for `use`.
`scope`	Applies to every fault.
`safety`	Circuit-breaker limits: `max_cost_usd`, `max_failure_rate`, `failure_window`, `min_calls`, `max_duration_seconds`, `max_injections`.
`dry_run`	Default for runs
`stop_on_failure`	Default `true`
`schedule`	`{cron, timezone}`, or a cron string

Validation (lint)

agentic-chaos experiment validate experiments/*.yaml   # exits 0 or 1

Validation reports every problem it finds with its location, for example:

faults[0].type: unknown fault 'rate_limt_storm'; did you mean 'rate_limit_storm'?
faults[1].probability: 2 is out of range (must be between 0 and 1)
assertions[0]: unknown assertion 'max_retrys'; did you mean 'max_retries'?
schedule.cron: hour value 25 is outside 0-23

It checks:

the schema shape and unknown fields;
fault and assertion names, with suggestions;
constructor parameters and their numeric ranges;
parameters that only accept Python callables (such as degrade_fn), which an experiment file can't set;
scopes, safety limits, cron fields and time zones;
template parameters and recursive template use;
duplicate node names;
targets.

Targets are parsed, never executed. A file must exist, compile and define the function at its top level. A module must be locatable with importlib.util.find_spec, which imports the module's parent packages but never the module itself. Pass --no-target-check to skip target checks, and --base-dir DIR to resolve relative targets elsewhere.

Some findings are warnings rather than errors: templates that are never used, and schedules that can never fire (such as February 31st). --strict makes warnings fail too, and --json prints machine-readable results. Because nothing is executed, this is safe as a pre-commit hook.

From Python, validate_experiment(path_or_document) returns the issues without raising. load_experiment() raises ExperimentError, which lists every issue.

Running

agentic-chaos experiment run experiments/resilience-suite.yaml \
  --approve approve-escalation --save workflow.json \
  --save-report merged.json --save-aios run.aios.json
agentic-chaos chaos workflow run experiments/resilience-suite.yaml   # same command

Every run validates the experiment first.

Exit code	Meaning
`0`	The workflow passed
`3`	The run stopped at an unapproved `Suspend`
`1`	Any other outcome: failed, halted, aborted or rejected

After an exit 3, re-run with --resume-from GATE.

How gates are decided:

--approve NAME (repeatable) approves the named gate.
--approve-all approves every gate.
In an interactive terminal, any other gate prompts. --no-input never prompts, so unapproved gates suspend the run.

The merged AIOS Run status is:

Workflow	AIOS Run status
`passed`	`completed`
`suspended`	`running` (it can be resumed)
aborted	`cancelled`
anything else	`failed`

Scheduling

agentic-chaos embeds no scheduler. experiment schedule previews the next runs and prints an install snippet:

agentic-chaos experiment schedule experiments/resilience-suite.yaml --next 3
agentic-chaos experiment schedule experiments/resilience-suite.yaml --format github-actions

The cron parser supports:

five fields, with *, lists, ranges and steps;
month and weekday names, and 7 as Sunday;
@hourly, @daily, @weekly, @monthly and @yearly;
Vixie-cron day-of-month or day-of-week matching when both are restricted. As in Vixie cron, a day field that starts with * (including */2) counts as unrestricted.

Crontab output adds CRON_TZ for zones other than UTC. GitHub Actions schedules always run in UTC; the snippet warns when the experiment names a different zone. examples/github-actions-chaos-workflow.yml lints on every push and runs the suite nightly.

Topology fuzzer

fuzz_topology() explores rather than following a fixed plan:

from agentic_chaos import Scope
from agentic_chaos.workflows import fuzz_topology

report = fuzz_topology(my_graph, n=20, seed=0, scope=Scope(exclude_steps=["safety_check"]))
print(report.failing_nodes)

How it works:

A fault-free baseline run discovers the topology: from the TopologyTracker the target creates, or else from the step names it calls. You can pass topology= instead (a tracker, an AgentTopology, a report, or a path to one).
Each trial picks a node, weighted by 1 + its fan-out (the number of distinct nodes it sends to). Coordinators are therefore hit more often than leaf tools.
The trial picks a fault suited to the node type, scopes it to exactly that node, and runs the target once.

The default faults per node type are in DEFAULT_FAULTS:

Node type	Default faults
tool	`tool_failure`
memory	`memory_corruption`
agent or router	`rate_limit_storm`, `silent_degradation`
anything else	`rate_limit_storm`

Pass faults= as a list, or as a {type: [...]} mapping, to choose your own.

A trial passes when all of these hold:

the target returns;
recovered(output), if given, accepts the output;
every contract passes.

Node eligibility uses only the step parts of scope. Provider limits apply when the node-scoped fault decides whether a call is in scope.

A trial whose fault never fires is reported as reached=False. That happens, for example, when a node is only a caller label with no wrapped call site. The same seed gives the same placements.

Targets must not pin faults=[...] at call sites to names outside the pool, and must reset their own external state per run. fuzz_topology() complements find_breaking_point(): one explores the whole graph, the other searches one parameter.

From the CLI (exits 1 when a reached trial failed):

agentic-chaos agent fuzz my_graph.py --n 20 --seed 0 --exclude-step safety_check --save fuzz.json

Evidence and ecosystem compatibility

Native evidence. Native event shapes and IDs are unchanged. Workflow provenance (detail.workflow) and the ChaosReport.workflow summary are additive, and older reports still load.
AgenticLens reads workflow reports and merged reports as Workflow JSON.
Control Tower counts merged native events.
MCP exports merged reports with the workflow summary in the extension and discovers faults through the existing registry. Workflows and experiments are Python and CLI orchestration, not new MCP tools. Hosted HTTP still rejects script execution.
AI Operations Specification. Merged exports validate against the AIOS v0.4 schema and semantic checker. The workflow summary is a producer-local extension, not a normative AIOS Workflow object.
Agentic Evals keeps ownership of scores and release gates. Workflow statuses and contracts are per-run resilience checks.
Agentic Sidecar 0.8.5 widens its optional extra to admit Chaos 0.8.

Health checks, workflow statuses and fuzz outcomes are resilience evidence. They don't prove containment: injecting into Sidecar/MCP gate, approval and evidence paths is still open roadmap work.