Skip to main content

Python Example: Halt an Agent on an Incident

This example turns a security incident into a halt on an approved Organization-owned agent. It persists the request before sending it, saves the accepted intervention ID, and reads both intervention and effective target evidence. It uses Python 3.12 or later and the standard library on Linux or macOS.

Complete the scoped identity and deployment setup in the External Security Halt guide first. The identity needs both security:halt and security:read. The example places a real hold; exercise it with an idle test agent before wiring it into your incident system.

Download and configure​

Download the Python handler, asset map and incident example into one directory. In a repository checkout, they live under docs/static/examples/security-halt/.

Maintain assets.json as trusted integration configuration. Its keys are asset identifiers your incident system knows; values are approved DT agent UUIDs:

{
"security-test-agent": "8fe80423-34d1-4f17-95ad-aac7597cd201"
}

The incident supplies an asset key, never a DT agent UUID. An unknown asset or an extra incident field is rejected before any HTTP request. The service identity's explicit allowlist and current Organization ownership remain authoritative on the server.

Create incident.json with a stable event ID for this specific containment action:

{
"event_id": "detector-a:incident-4812:halt-v1",
"asset_id": "security-test-agent",
"reason": "Credential theft alert linked to this approved asset",
"incident_reference": "INC-004812"
}

Replace the sample UUID and incident values. Namespace event IDs across detectors using this identity. Keep the event ID, asset mapping, reason and optional reference unchanged when retrying. A deliberate new containment action after release needs its own new event ID.

Create a private state directory on persistent local storage:

mkdir -m 700 security-halt-state

Use one directory per deployment and service identity. The handler binds its SQLite database to the HTTPS origin and stable identity UUID and refuses to reuse it with a different binding. Use the identity UUID recorded during credential provisioning, rather than the credential UUID embedded in the token. Token rotation retains the identity; creating a replacement identity changes the server's deduplication namespace.

--identity-id is a local configuration safeguard. The server authenticates the token's actual identity; supply a token belonging to the original identity on every retry.

The database contains incident text, immutable request bodies and evidence snapshots in plaintext, with owner-only filesystem access. Retain it and the original incident/configuration files for retry recovery; protect and back up the state with your incident data. Do not use an ephemeral container filesystem or put the state directory in a shared checkout. The bearer token is never written to the database.

Run an incident​

Replace the deployment origin and stable service identity UUID:

python3 incident_handler.py \
--origin https://dt.example.com \
--identity-id 647676a1-a189-4a80-a699-12235a5d9a53 \
--asset-map assets.json \
--incident incident.json \
--state-dir security-halt-state \
--poll-seconds 120

The handler prompts for the bearer token without echo. For an automated run, inject DT_SECURITY_TOKEN through your secret manager's process environment. Do not pass the token on the command line or record it in incident files. The CLI prints selected identifiers, control evidence and a next action; failures omit server free text and retain a valid support error_id when available.

The client requires an HTTPS origin without credentials or a path, verifies the certificate using Python's default trust configuration, and never follows redirects. It connects directly; proxy environment variables are not applied. Use an origin and trusted CA configuration appropriate for the integration host.

The request and its normalized event ID commit locally before the first POST. Timeouts, disconnected responses, malformed JSON success responses and HTTP 500/502/503/504 receive at most three attempts, with one- and two-second delays. Every POST attempt uses the same persisted body and identity. If a response is still unavailable, the row remains pending; the remote hold may already exist.

Once an intervention ID is saved, rerunning the same command resumes status reads instead of sending another POST. A restart before the ID is saved sends the original request again; server deduplication returns the current intervention with 200 and replayed: true. Preserve the original identity when replacing a revoked token. Do not delete local state or invent another event ID to work around an uncertain result.

Interpret the result​

The handler polls until the intervention leaves accepted/enforcing, the scheduling budget expires, or 12 logical reads have occurred. Each logical read has at most three transport attempts. Delays grow from one to eight seconds. Socket operations use a timeout of at most ten seconds, reduced to the remaining polling budget. The budget is checked between requests and delays; it is not a strict wall-clock deadline for an in-progress HTTP exchange. A final effective target read has its own bounded retry allowance.

Successful output includes intervention_id, the latest intervention state, independent enforcement evidence, and a control_state snapshot. Both complete responses are saved in SQLite. These are separate reads taken at different times; continue monitoring if another halt or release races with them.

Exit codeResultNext action
0enforced_hold_active: this intervention and the effective target report enforcement, active barriers, verified runtime acknowledgement and no unresolved tracked work.Record the evidence and hand the incident to an authorized human. The hold remains active.
2human_review_required: partial enforcement, failure, released state, exhausted polling reads/budget, another hold's unresolved work or differing evidence.Review the saved snapshots and DT. Arrange follow-up reads or operator investigation.
1Input, storage, HTTP, authentication or response-validation failure. Acceptance may be uncertain.Resolve the named error using the retained request and identity. Existing holds remain effective.

An exhausted request retry allowance, or a polling budget that expires before a read can start, returns 1. Saved evidence is retained for follow-up. To inspect event/ID correlations without changing the database:

import sqlite3
from contextlib import closing

with closing(sqlite3.connect("file:security-halt-state/incidents.sqlite3?mode=ro", uri=True)) as db:
for event_id, intervention_id in db.execute("SELECT event_id, intervention_id FROM incidents"):
print(event_id, intervention_id or "acceptance uncertain; retry the original request")

Automatic retries stop for these errors:

ErrorRecovery
asset_not_approvedVerify asset ownership and the trusted map before allowing containment.
local_event_conflictThe saved event has different target/text semantics. Resolve the discrepancy from the original incident; the handler has sent no replacement request.
external_event_id_conflict / HTTP 409Resolve the server's original event semantics. Other 409 cases appear as request_conflict; check runtime availability and ownership with the operator.
authentication_failed / HTTP 401Review credential revocation, identity enablement and deployment gates. Rotate within the original identity when appropriate, then rerun with its replacement token. Revocation does not release an existing hold.
permission_denied / HTTP 403Provision the required security:halt and security:read permissions.
target_or_intervention_not_found / HTTP 404Check the mapped UUID, Organization and current explicit scope; these failures intentionally share a response.
state_binding_conflictUse the original origin/identity for this state directory. A different identity cannot replay the original identity's event.
TLS, redirect or evidence-validation errorInvestigate the origin, deployment compatibility and trust configuration. Retain the request and any saved ID; do not weaken TLS or assume the hold failed.

The example stops for human review at partially_enforced and failed; it does not keep retrying enforcement or repeating external work. Unknown external outcomes remain evidence, including after human release. Missing worker acknowledgement does not prove that a process is absent.

Human recovery and follow-up​

Use the human release and final verification steps with the saved intervention ID. Release requires DT permissions, recent authentication and the applicable name/outcome confirmations. The handler has no release, enforcement-retry or agent-resume operation.

Rerun the same command after release to save fresh evidence. It returns human_review_required with state: released; check the effective target's held, active_intervention_ids and unresolved work to see whether another hold or unknown outcome remains. A released intervention never authorizes automatic continuation. Any desired execution is a deliberate fresh action through ordinary admission and approval checks.

For a deployed adapter, call this logic from your trusted incident ingestion path, retain the same per-identity state, and route nonzero results into your incident system's human queue. This example supplies the HTTP and durable retry workflow; provider-specific alert authentication, incident ingestion and remote validation belong to that adapter.