Browse documentation
Threat model & security requirements
Trust boundaries, misuse cases, operator permissions, agent privileges, and the failure modes the design must address.
Engineering review draft. The design, API, and AWS integrations described here are proposed; operating results and response-time commitments have not yet been validated.
On this page
Security objective and assumptions
An authorized operator should be able to request containment of an enrolled workload without obtaining cooperation from its code or model. The system must accurately report the evidence it has, retain quarantine after incomplete outcomes, and reserve recovery for a human decision.
Assume the agent controls its process, input, local files, and task-role credentials. It may ignore signals, make repeated tool requests, request child work, and attempt to exploit any granted permission. Do not assume agent-generated logs or self-reported status are trustworthy.
The prototype trusts AWS isolation, the selected identity provider, authorized control-plane administrators, the approved provisioning pipeline, and the protected registry. Compromise of these roots of trust is a separate response scenario, not a capability the prototype claims to defeat. Fargate task isolation is relevant to the selected boundary; it does not make arbitrary containers or unrelated runtimes equivalent. [S5]
Assets and trust boundaries
| Asset or boundary | Protection needed |
|---|---|
| Operator identity to control API | Authentication, configured MFA assurance, action authorization, replay protection, and workload-generation binding. |
| Control account to workload account | Narrow connector role, exact trusted principal, environment binding, and no agent role assumption. |
| Registry and quarantine latch | Strongly consistent gate reads, conditional writes, protected provisioning, and no agent write access. |
| Launcher to ECS | Reservation before launch, generation fencing, exact ARN registration, and reconciliation of in-flight calls. |
| Agent to tool gateway | Workload identity mapping, per-request admission, default denial on uncertainty, and no alternate supported tool path. |
| Evidence and recovery records | Agent-inaccessible writes, scoped reader access, redaction, and an auditable approval chain. |
Threat and control register
| ID | Threat | Required control | Verification drill |
|---|---|---|---|
| T-01 | Agent ignores a stop instruction or termination signal. | Infrastructure task stopping; observe final task state without trusting the agent. | D-01 |
| T-02 | Agent launches replacements or races a stop with a child launch. | Exclusive launcher, durable fence, launch reservations, and repeated inventory reconciliation. | D-02, D-03 |
| T-03 | Agent uses fresh or copied credentials after intervention. | Dedicated generation role, persistent deny policy, blocked role escalation, and gateway latch. | D-04 |
| T-04 | Agent bypasses the gateway through another endpoint or network path. | Restricted routing, endpoint policies, scoped task role, and enrollment rejection on unsupported paths. | D-05 |
| T-05 | API caller substitutes an account, task ARN, role ARN, or another workload ID. | Server-owned registry mapping; action authorization; connector allowlists; strict request schemas. | D-06 |
| T-06 | Compromised operator or stolen token triggers a harmful shutdown. | MFA-backed identity policy, short sessions, least privilege, scoped confirmation, audit, and a documented operator-revocation path. | D-07 |
| T-07 | Agent tampers with evidence or forges a success report. | Independent observers and evidence writes outside agent permissions; preserve observation provenance. | D-08 |
| T-08 | AWS throttling, permission drift, or connector outage interrupts a stop. | Bounded retries, durable quarantine, per-path results, visible incomplete outcomes, and reconciliation. | D-09 |
| T-09 | Client retry or worker restart creates conflicting operations. | Durable application idempotency and conditional state transitions; idempotent connector actions. | D-10 |
| T-10 | Recovery restores a compromised old identity or replays an approval. | Review bound to the verified incident/generation; single-use permit; fresh role and generation; old deny remains. | D-11 |
| T-11 | An already admitted remote action finishes after the stop. | Track controlled in-flight work, drain or account for it, and disclose the boundary around external side effects. | D-12 |
| T-12 | Inventory omits a child or ownership tags disappear/change. | Persist enrolled exact ARNs before intervention; reconcile the designated boundary; never infer coverage from tags alone. | D-03, D-08 |
Security requirements
- SR-01: Agent credentials cannot modify the API, workflow, connector roles, registry, launch fence, gateway policy, evidence store, or recovery approvals.
- SR-02: API authorization resolves identity, environment, permitted action, and workload from trusted configuration. Request bodies never select arbitrary AWS resource ARNs.
- SR-03: The identity integration must verify issuer, audience, signature, expiry, and configured authentication assurance. Missing or unsupported MFA assurance fails authorization. The identity provider and exact claim mapping remain implementation decisions.
- SR-04: Stop acceptance requires durable quarantine and a recorded operation/outbox entry. Dispatch failure cannot undo quarantine.
- SR-05: Gateway and launcher decisions read authoritative gate state without a positive authorization cache. Unavailable or ambiguous state denies new work.
- SR-06: Gateway denial covers new admitted requests in the supported adapter. It does not retroactively cancel a request already dispatched to an uncontrolled system.
- SR-07: Connector credentials are scoped to enrolled resources. Broad discovery permissions, where unavoidable, do not imply permission to stop every discovered task.
- SR-08: Per-generation task-role quarantine uses a persistent deny, not only a cutoff for older sessions. Old identities remain quarantined after recovery. AWS's session-revocation mechanism alone does not restrict all later sessions. [S2]
- SR-09: Required verification checks use provider observations and controlled probes. An agent heartbeat or accepted StopTask call cannot independently establish success.
- SR-10: Recovery review approval records permission to provision a new generation. It does not clear the stop latch or restore the previous role.
- SR-11: Evidence excludes secrets and unnecessary prompt, output, and personal data. Store hashes, identifiers, outcomes, provenance, and bounded reason fields where sufficient.
- SR-12: Unknown inventory, unexpected activity, or stale observations prevent a current complete-containment assertion. Preserve previous evidence rather than rewriting its history.
Failure behavior and residual risk
If the identity provider or control API is unavailable, the normal human command path may be unavailable. A separately governed AWS operator procedure is an additional access path, not protection against an AWS-wide management outage. Gateway fail-closed behavior also introduces an availability tradeoff during control-state failures.
Changing a security-group rule is not proof that every existing connection was terminated; tracked flows can persist after a rule change. The design therefore does not treat a group swap as a verified network cutoff. [S6]
An agent with uncontrolled outbound access, alternate credentials, an unknown child workload, or authority to change account policy is outside the validated prototype boundary. These conditions require enrollment rejection or an explicit incomplete-coverage report. The tool also creates denial-of-service risk if its own operator or control account is compromised; that risk belongs in incident drills and operating decisions.
References
[S2] Revoke IAM role temporary security credentials.