Skip to content

Native Evals schema validation (Control Tower 0.6.1, synced to Evals 0.9.0 in 0.7.1)

Tower validates native reports and gate decisions against the published Evals Draft 2020-12 JSON Schemas: Evals 0.7.2 from Tower 0.6.1, and Evals 0.9.0 since Tower 0.7.1. The 0.9.0 documents add optional judge, live-target and target_status fields; see the 0.9.0 note. Exact upstream documents are bundled as package data, generated by scripts/sync_evals_schemas.py through report_schema() / gate_decision_schema(). Validation is offline and requires no runtime Evals or Lens installation. CI compares the copies with accessors from the pinned agentic-evals==0.9.0 development dependency (0.7.2 before 0.7.1).

Unknown fields remain allowed, including nested fields. Unsupported report schema_version markers are rejected explicitly. JSON Schema format annotations are not enforced; downstream model readers retain their existing datetime checks. Caller-supplied $ref strings are ordinary artifact data; no caller schemas or remote resources are resolved. Diagnostics name the failing path and constraint without echoing invalid values.

Schema validity is structural, not AIOS conformance, score arithmetic verification or evidence integrity. A structurally valid failed gate stays failed. Evals owns scoring and release policy; consumers do not invent new verdicts.

Ingestion behavior and migration

Every Evals report submitted through summarize_evidence, the evidence store, HTTP, CLI or snapshot ingestion is checked before summary projection. A supplied gate must validate too. Invalid reports or gates produce availability: "unavailable" with an empty details projection and a diagnostic; they cannot supply a passing gate for recovery verification.

Previously accepted summary-only objects are not complete native reports. Send the complete EvaluationReport.model_dump(mode="json"), including cases and the required summary average_latency_ms. Extra fields can be retained by the producer even though Tower stores only its operational projection. Use the complete report example and gate example, or run the existing examples/incident_recovery.py producer walkthrough.

The optional agentic-evals extra now requires >=0.7.2,<1; no producer package is required to start Tower or validate native JSON. The base package adds jsonschema for contract validation. Persisted Tower projections are not native reports and must not be submitted back as producer artifacts.

When a writer omits defaulted schema_version, Tower interprets it as 1.0. An omitted producer created_at is projected as null (unknown); Tower does not fabricate a producer timestamp. The separately attributed link.observed_at continues to drive evidence freshness. Required-field errors identify the missing field, including nested case/score paths.