Skip to content

v0.5: streaming, provider patches, and profiles

v0.5 preserves the existing ChaosReport / chaos_events JSON contract. Provider SDKs remain optional: pip install agentic-chaos[providers] installs OpenAI, Anthropic, and Google Gen AI together; install an individual SDK instead if you only use one provider. Python 3.10 uses tomli; later versions use tomllib.

Streaming

from agentic_chaos import StreamCutFault, chaos_session, chaos_stream

with chaos_session([StreamCutFault(after_chunks=2)]) as session:
    chunks = list(chaos_stream(iter(["Hello", " world", "!"]), step_name="answer"))
assert chunks == ["Hello", " world"]
Fault Parameters Behavior
StreamCutFault after_chunks=1 Ends normally after N delivered chunks, without reading ahead
StreamHangFault after_chunks=0, delay_seconds=1 Freezes at N chunks, then raises StreamHangError
SlowTTFTFault delay_seconds=1 Delays the first delivered chunk
SlowChunksFault delay_seconds=1 Delays each delivery after the first chunk

All accept probability=1.0 and seed=None. Probability is sampled once per stream. chaos_astream() provides the same behavior for async iterators using cancellable async sleeps. Stream faults belong on chaos_stream() / chaos_astream(), rather than chaos_call(). Consume the stream inside the session, and explicitly close() / aclose() on early abandonment. Wrappers own and close their source on exhaustion, injected failure, or explicit close.

Cut events use outcome="degraded", hang events "errored", and delay events "delayed". detail records delivered chunk count, configured cut point, and delay. Events are emitted when the stream is affected, not when it is created. Slow faults emit once per stream; empty streams don't produce delay events. A cut at exactly N chunks still records the configured cut without consuming an extra chunk to discover whether the upstream stream would have ended naturally. Hang delays are bounded simulations, not infinite waits or real socket timeouts.

Provider patching

from openai import OpenAI
from agentic_chaos import StreamCutFault, chaos_session, patch_openai

client = OpenAI()
with chaos_session([StreamCutFault(after_chunks=2)]) as session:
    patch_openai(client)
    with client.chat.completions.create(
        model="your-model", messages=[{"role": "user", "content": "Hello"}], stream=True,
    ) as stream:
        for chunk in stream:
            print(chunk)
# Original methods are restored, including when the application raises.

Pass an existing client explicitly. Patches affect that instance and the current session context. They leave unrelated clients and other contexts alone. A patch requires an active session and exactly one selected fault. With multiple faults configured, use patch_openai(client, faults=["stream_cut"]) to select one. Explicit instances must be the same instances configured in the session. ProviderPatch.close() allows earlier restoration; repeated close is safe. Duplicate patches are rejected and partially installed patches are rolled back.

Helper Patched SDK methods
patch_openai(client) chat.completions.create, responses.create
patch_anthropic(client) messages.create, including stream=True
patch_google(client) models.generate_content, models.generate_content_stream, plus client.aio.models equivalents

Sync and async SDK clients are supported. Provider stream wrappers preserve iteration, context management, source attributes, and close methods. Streaming faults pass non-streaming calls through. Anthropic messages.stream() and OpenAI high-level .stream() accumulators are outside the patch surface; use the listed create(..., stream=True) methods. Google support targets google-genai, not the legacy google-generativeai package. Provider factories and cloned clients need their own explicit patches.

AuthErrorFault(status_code=401) supports 401/403 and raises AuthError; ContextLengthFault() raises ContextLengthError with status 400 and code context_length_exceeded. These are package exceptions carrying ChaosEvent, not SDK-specific exception classes. They prevent the real call from executing.

chaos_acall() applies existing call faults to async functions. Blocking legacy fault delays run in a worker; the actual provider coroutine stays on the caller's event loop. Cancellation cancels pending provider work and discards the isolated event buffer. A bounded synchronous fault delay can finish in its worker after cancellation. Stateful fault instances should not be shared across concurrent calls or sessions.

Probability and profiles

Every fault accepts probability from 0 to 1 and an optional seed. Probability zero passes through without recording an event or advancing fault counters; one retains deterministic v0.4 behavior. Sampling occurs at injection entry points, not when calling the low-level trigger() implementation directly. Existing multi-fault selection remains explicit; probabilities don't choose which fault applies to a call site.

[profiles.production-like]
faults = ["rate_limit_storm", "token_timeout"]
probability = 0.1
seed = 42

[profiles.production-like.parameters.token_timeout]
hang_seconds = 0.2
agentic-chaos chaos run examples/chaos_streaming_demo.py \
  --profile streaming-demo --profile-file examples/chaos.toml --save report.json
agentic-chaos agent run my_agent.py --profile production-like

Use exactly one of --inject and --profile. The default profile file is chaos.toml in the working directory. load_profile(name, path) returns fresh fault instances for Python callers. Unknown profiles, names, fields, duplicate faults, and invalid parameters fail before executing the target script. Per-fault parameter tables may override the profile probability and seed. Applications using multiple configured faults must still select a fault with chaos_call(..., faults=["token_timeout"]) (or the corresponding stream API).

Sibling compatibility

  • AgenticLens: existing workflow parsing, attach_events(), and ChaosImpactRecommender consume the new names with existing outcome semantics. Fidelity weighting and a drift recommender remain separate roadmap work.
  • MCP: the fault inventory reads FAULT_REGISTRY dynamically. Its script runner accepts the new names; scripts must use the appropriate stream API. MCP's request schema does not expose CLI profiles or provider patch configuration.
  • Control Tower: native chaos_report ingestion reads the unchanged report shape and event count. Producer outcomes remain separate from fleet health.
  • AI Operations Specification: v0.5 introduces no new shared object or canonical fields. Native ChaosReport is not a canonical AIOS export; structured experiment reports and explicit AIOS exports remain unimplemented v0.2.x work.
  • Evals / Sidecar: no new runtime coupling or dependency is required.

CI tests the core package on Python 3.10–3.13, real optional SDKs without provider network calls, and actual Lens, MCP, and Tower sibling checkouts. The optional agenticlens extra remains a development sibling source until its compatible published package is available; v0.5 doesn't silently add any sibling dependency.

Run the offline examples with uv run python examples/chaos_streaming_demo.py and uv run python examples/chaos_provider_patching_demo.py.

Offline v0.5 demo

Safety rails ship in 0.5.2; see the safety guide.