# Safegrade telemetry for Auga

> Machine-readable brief for Auga and its agents. Safegrade is a synthetic company: every company,
> person, agent and trace described here is fabricated. All paths below are relative to this site.

Auga reads the OpenTelemetry traces a company's AI agents already emit, finds the root cause of what
went wrong and ranks the fixes. To show that, it needs a customer with agents, telemetry and a real
problem to find. Safegrade is built to be that customer. Everything here is fabricated, so it can be
shown to anyone.

## What Safegrade is

A fictional Swedish credit-information company (company search, board, debts, annual figures, a
1–12 score, certificates, supplier monitoring). The public site runs on a synthetic register of 9 999
companies, served from SQLite through the site's own API (`/api/search/?q=`,
`/api/company/<orgnr>/`). Nine AI agents run the business behind it.

## Why it suits Auga

- **Credit scoring is high-risk under the EU AI Act** (Annex III, point 5(b), for sole traders).
  Scorekeeper therefore needs logging, human oversight and traceability: exactly what the EU AI Act
  lens collects.
- **The agents form a chain.** Each reads what earlier agents wrote, so a fault surfaces two or three
  agents later, in a different place. That is the case where correlation misleads and causal
  analysis pays off.
- **Every agent has a human-oversight point and a measurable outcome**, which map directly to the
  Oversight Inbox and to the outcome chosen in Auga's onboarding.

## The nine agents

Shown on the home page; each agent has a page at `/agents/<id>/` (for example
`/agents/scorekeeper/`), and `/agents/` shows how they depend on each other.

| Agent | Reads from | AI Act | Outcome to optimize | Auga lenses |
|-------|-----------|--------|---------------------|-------------|
| Registry Sync | the official register | minimal | Register freshness | Operate, Agent Health |
| Annual Report Reader | Registry Sync | minimal | Extraction accuracy | Agent Health, Investigation |
| Debt Watch | Registry Sync | limited | Time to record | Oversight Inbox, Operate |
| Board Screener | Registry Sync | limited | Screening precision | Oversight Inbox, EU AI Act |
| Scorekeeper | Registry Sync, Annual Report Reader, Debt Watch | **high** | Score disputes | EU AI Act, Investigation, Oversight Inbox |
| Certification Desk | Scorekeeper | limited | Decision accuracy | Oversight Inbox, EU AI Act |
| Supplier Monitor | Scorekeeper, Debt Watch, Board Screener | limited | Alert precision | Cost Optimization, Operate |
| Trust File Publisher | Registry Sync, Annual Report Reader, Debt Watch | minimal | Profile freshness | Agent Health, Operate |
| Customer Desk | Scorekeeper, Certification Desk | limited | Resolution without hand-off | Investigation, Oversight Inbox |

## Telemetry

`/telemetry/safegrade-agents.otlp.json`. Generated deterministically by
`scripts/generate-agent-telemetry.mjs` in the Safegrade repository (`npm run agents:telemetry`).

- **Format:** OTLP/JSON, one `ExportTraceServiceRequest` on a single line. The OpenTelemetry
  Collector's `otlpjsonfile` receiver reads it as-is, and it can be POSTed to any OTLP/HTTP endpoint
  at `/v1/traces`.
- **Window:** seven days, 2026-09-18 to 2026-09-24, about 1 900 spans.
- **Resources:** one per agent. `service.name` is `safegrade-<agent>`, `service.namespace` is
  `safegrade`, and `safegrade.synthetic` is `true`.
- **Spans:** one trace per agent run, following the OpenTelemetry GenAI semantic conventions:
  - root `invoke_agent <name>` with `gen_ai.agent.id`, `gen_ai.agent.name`, `gen_ai.request.model`,
    token totals and the agent's `safegrade.*` business attributes;
  - children `chat <model>` (with `gen_ai.usage.*`) and `execute_tool <tool>` (with
    `gen_ai.tool.name`).
- **Causal edges:** each run's root span has a span link to the latest run of every upstream agent,
  with `safegrade.link.reason = reads_output`.

The `safegrade.*` attributes per agent are listed on each agent's page.

## The planted incident

On **2026-09-22**, Registry Sync's `fetch_register_delta` tool call is slow (about 100 s instead of
under a second) and returns the previous day's file. The run still succeeds, so no error fires.

| Where it shows | Signal |
|----------------|--------|
| Registry Sync | `safegrade.register.lag_hours` = 31 (normally 6–14), few records changed |
| Scorekeeper | every score drops 3 steps; `safegrade.input.financials_age_days` ≈ 365–400 (normally < 40); sign-off required |
| Certification Desk | every decision is `refuse` |
| Supplier Monitor | 6–11 alerts per run instead of 0–2 |
| Trust File Publisher | profiles served about a day old |
| Customer Desk | mostly `dispute`, most conversations handed off |

**Expected answer:** the root cause is Registry Sync's stale register delta, not Customer Desk or
Supplier Monitor, where the damage is visible. The recommended fix is to alert on
`safegrade.register.lag_hours` and to stop downstream agents from running on stale input.

## What we expect back

[`/auga-expected.json`](/auga-expected.json) is the answer Safegrade expects Auga to produce from this
telemetry, in a small JSON contract (`safegrade.auga-expected/v1`):

- `deviation`: what went wrong, with each symptom's baseline and observed value;
- `root_cause`: the agent, span and signal, with lead time before the first customer impact;
- `causal_chain` and `not_the_cause`: how it spread, and which visibly affected agents are innocent;
- `fixes`: ranked by value, each tagged with the Auga lens that would carry it;
- `ai_act`: evidence for Scorekeeper, the high-risk system, requirement by requirement.

Every figure in it is computed from the telemetry file. The site's **Auga side panel** (the Auga button
in the navigation bar) replays the telemetry in the browser and shows this answer next to the evidence:
root cause, traces with their upstream links, per-agent health, and EU AI Act evidence.

## Next steps

1. **Replay into Auga.** Send the file to Auga's collector (`otlpjsonfile` receiver, or an OTLP/HTTP
   POST) and check that the causal graph finds Registry Sync.
2. **Live telemetry.** Replace the generator with small agent stubs that emit the same spans
   continuously through the OpenTelemetry SDK, so Auga sees a stream rather than a file.
3. **More incidents.** Each agent's page lists a second failure mode (for example the iXBRL sign
   flip in Annual Report Reader). Planting those gives Auga a set of cases to be tested against.