All agents

Checks each applying company against the certificate criteria (score, filing timeliness, no open remarks) and drafts the award or the refusal with its reasons.

Runs per day

2

p95 run time

10.8 s

Tokens per day

21,885

Failed runs

0.0 %

From the sample telemetry, 2026-09-18 – 2026-09-24, which includes the incident on 2026-09-22.

Facts

service.name

safegrade-certification-desk

Model

claude-sonnet-5

Runs

On each application, and a monthly re-check of all certificate holders

Tools

load_applicationcheck_criteriadraft_decisionissue_certificate

Outcome to optimize

Decision accuracy: No certificate revoked within 90 days of issue (safegrade.certificate.decision)

Human oversight

A person approves every award and every refusal.

EU AI Act

Recommends a decision; a person decides. Relies on Scorekeeper, a high-risk system.

Reads from

Scorekeeper

Read by

Customer Desk

Auga lenses

Oversight Inbox, EU AI Act

What can go wrong

• Symptom: Certificates refused for companies that meet the criteria

  Root cause: The score it read was the stale one from Scorekeeper’s incident run

Telemetry

Each run is one trace. Besides the standard gen_ai.* attributes, the root span carries:

Shape of one run:

{
  "name": "invoke_agent Certification Desk",
  "resource": {
    "service.name": "safegrade-certification-desk",
    "service.namespace": "safegrade"
  },
  "attributes": {
    "gen_ai.operation.name": "invoke_agent",
    "gen_ai.provider.name": "anthropic",
    "gen_ai.agent.id": "certification-desk",
    "gen_ai.agent.name": "Certification Desk",
    "gen_ai.request.model": "claude-sonnet-5",
    "safegrade.orgnr": "…",
    "safegrade.certificate.level": "…",
    "safegrade.certificate.decision": "…"
  },
  "children": [
    "chat claude-sonnet-5",
    "execute_tool load_application",
    "execute_tool check_criteria",
    "execute_tool draft_decision",
    "execute_tool issue_certificate",
    "chat claude-sonnet-5"
  ]
}

Download the full week (OTLP/JSON)