Native Evals schema validation (Control Tower 0.6.1, synced to Evals 0.9.0 in 0.7.1)
Tower validates native reports and gate decisions against the published Evals
Draft 2020-12 JSON Schemas: Evals 0.7.2 from Tower 0.6.1, and Evals 0.9.0 since
Tower 0.7.1. The 0.9.0 documents add optional judge, live-target and
target_status fields; see the 0.9.0 note. Exact upstream documents are bundled as
package data, generated by scripts/sync_evals_schemas.py through
report_schema() / gate_decision_schema(). Validation is offline and requires
no runtime Evals or Lens installation. CI compares the copies with accessors
from the pinned agentic-evals==0.9.0 development dependency (0.7.2 before 0.7.1).
Unknown fields remain allowed, including nested fields. Unsupported report
schema_version markers are rejected explicitly. JSON Schema format annotations
are not enforced; downstream model readers retain their existing datetime checks.
Caller-supplied $ref strings are ordinary artifact data; no caller schemas or
remote resources are resolved. Diagnostics name the failing path and constraint
without echoing invalid values.
Schema validity is structural, not AIOS conformance, score arithmetic verification or evidence integrity. A structurally valid failed gate stays failed. Evals owns scoring and release policy; consumers do not invent new verdicts.
Ingestion behavior and migration
Every Evals report submitted through summarize_evidence, the evidence store,
HTTP, CLI or snapshot ingestion is checked before summary projection. A supplied
gate must validate too. Invalid reports or gates produce availability:
"unavailable" with an empty details projection and a diagnostic; they cannot
supply a passing gate for recovery verification.
Previously accepted summary-only objects are not complete native reports.
Send the complete EvaluationReport.model_dump(mode="json"), including cases
and the required summary average_latency_ms. Extra fields can be retained by
the producer even though Tower stores only its operational projection.
Use the complete report example and
gate example, or run the existing
examples/incident_recovery.py producer walkthrough.
The optional agentic-evals extra now requires >=0.7.2,<1; no producer package
is required to start Tower or validate native JSON. The base package adds
jsonschema for contract validation. Persisted Tower projections are not native
reports and must not be submitted back as producer artifacts.
When a writer omits defaulted schema_version, Tower interprets it as 1.0.
An omitted producer created_at is projected as null (unknown); Tower does not
fabricate a producer timestamp. The separately attributed link.observed_at
continues to drive evidence freshness. Required-field errors identify the missing
field, including nested case/score paths.