An air-gapped agentic AI architecture is a locally operated system in which models, orchestration, tools, data, and operational controls run within an approved isolated environment. The practical goal is not simply to remove public-cloud calls. It is to give a program explicit control over what the agent can access, what actions it can request, how outputs are evaluated, and how software and models enter or change the environment.
That distinction matters for defense and intelligence teams. Isolation can reduce external dependencies and constrain data movement, but it does not establish trust among internal services. NIST’sZero Trust Architectureguidance says trust should not be granted solely because of network location. An isolated agent system still needs authenticated users and workloads, least-privilege authorization, protected data flows, and evidence of what occurred.
What should an air-gapped agentic AI architecture contain?
The architecture should contain seven separable layers. A program may deploy them on one approved platform or distribute them across managed infrastructure, but each layer should have a defined responsibility, interface, owner, and failure behavior.
1. User and mission interface
The interface receives requests, presents sources and results, records user decisions, and makes system limitations visible. It should reflect the mission workflow rather than expose every platform capability. For higher-consequence uses, the interface also needs clear approval, rejection, correction, and escalation paths so human responsibility is not hidden behind a conversational experience.
2. Identity and policy
Identity connects users, services, tools, and data permissions. Policy decides which combinations are allowed for a particular task and context. Network placement alone should not authorize a model service to retrieve a collection or permit an agent to invoke a tool. TheDoD Zero Trust Strategysimilarly focuses on users, assets, and resources rather than relying on a static perimeter.
3. Agent control plane
The control plane contains the workflow graph, prompt and policy versions, session state, tool-selection logic, limits, and human gates. It should constrain the agent’s possible actions instead of assuming a model will consistently follow natural-language instructions. An open-source agent platform can provide this layer, but its interfaces, dependencies, configuration, and extension model must be evaluated for the target environment. Our guide toevaluating open-source agent platforms for air-gapped usecovers that decision in more detail.
4. Inference services
Inference services load and execute the program’s approved models. Keep the agent application behind a stable internal inference interface so the team can compare or replace models without rewriting the workflow. Model choice should follow the mission task, hardware, latency, context, licensing, evaluation results, and sustainment burden—not a blanket preference for one model family.
5. Tools and integrations
Tools are the point where an assistant becomes an agent. Retrieval, search, calculation, document generation, workflow APIs, and other actions should be exposed through narrow adapters with typed inputs, authorization checks, timeouts, bounded outputs, and auditable results. Read-only tools are a safer starting point. Any state-changing action should have a mission-specific approval rule and a defined rollback or recovery path.
6. Data and knowledge
Data services may include approved document stores, metadata catalogs, retrieval indexes, databases, and temporary working state. Preserve provenance between a generated answer and the material used to produce it. Access should be enforced at the source or retrieval layer rather than trusted to a prompt. Retention and deletion rules should cover prompts, retrieved passages, outputs, caches, and derived indexes.
7. Evidence and operations
Observability should record enough information to reconstruct a run: authenticated actor, system and policy versions, model, tool requests, approvals, outcomes, timing, and relevant errors. Logs require their own access and retention rules; indiscriminate capture can create a new concentration of sensitive information. This layer also supports health monitoring, resource use, incident response, backup, recovery, evaluation, and configuration management.
How should the layers communicate?
Use explicit, versioned interfaces and make the permitted data flows inspectable. A request should move from an authenticated interface to a policy decision, then to bounded orchestration. The agent may call an approved inference endpoint or tool, while retrieval applies source permissions and evidence services record the event. The answer returns with citations, limitations, and any required human decision.
This separation limits accidental coupling. It also makes testing more precise: teams can evaluate a model independently, test whether policy blocks an unauthorized tool call, or verify that a source-permission change reaches the retrieval layer. NIST SP 800-207A’scloud-native zero trust modelemphasizes application and service identities in addition to network controls. Although an air-gapped deployment has different connectivity conditions, that principle remains useful for its internal services.
| Boundary | Decision to make | Evidence to retain |
|---|---|---|
| User to agent | Who may request which workflow? | Identity, role, request, policy result |
| Agent to model | Which model and configuration are approved? | Model identifier, parameters, prompt and policy version |
| Agent to tool | Which action is permitted under what approval? | Tool input, authorization, approval, output and error |
| Agent to data | Which sources may this actor and workflow retrieve? | Query, source identifiers, access decision, provenance |
| Transfer boundary | What artifact may enter or leave the environment? | Origin, review, signature or digest, destination, custodian |
How does software and model supply-chain control fit the architecture?
An isolated environment still changes. Models, runtime packages, containers, operating-system updates, evaluation sets, policies, and mission data may cross a managed transfer boundary. The architecture therefore needs a staging process outside or adjacent to production: acquire from an approved source, record provenance and licensing, scan and inspect, verify integrity, evaluate, approve, transfer, and confirm the installed state.
NIST’sSecure Software Development Frameworkprovides a common set of secure development practices for producers and acquirers. NIST SP 800-53 also organizes controls forconfiguration, system integrity, media protection, and supply-chain risk. Programs should map their actual transfer and authorization procedures to applicable policies and controls rather than treating this reference architecture as an authorization package.
Updates should be reversible and promoted through development, test, and production stages. A model update is not only a file replacement; it can change tool selection, refusal behavior, resource use, and task performance. The companionair-gapped AI model update lifecycleexplains how to plan that path.
What should be evaluated before mission use?
Evaluate the complete system, not just the model. The NISTAI Risk Management Frameworkconnects governance, context, measurement, and risk management. Its generative AI profile highlights risks that require system-level treatment, including confabulation, information integrity, privacy, information security, and value-chain integration.
- Mission performance: Does the bounded workflow improve the intended decision or task?
- Critical failures: Which errors, unsafe actions, or misleading outputs occur under representative and difficult cases?
- Control enforcement: Do identity, data, tool, and human-approval rules hold when inputs are malformed or ambiguous?
- Operational behavior: Are latency, capacity, recovery, logging, and resource use acceptable in the target environment?
- Change sensitivity: Which tests must be repeated when models, prompts, tools, policies, or knowledge sources change?
Begin with a boundedAI prototype test planand distinguish technical feasibility from evidence suitable for a pilot. The guide onprototype versus pilot evidencehelps programs choose the next learning stage without overstating what an early result proves.
What should a reference architecture deliberately leave open?
It should leave model, agent framework, inference runtime, data store, and deployment technology open until requirements and evaluation narrow the choice. Prescribing products too early can turn architecture into procurement by diagram. Instead, define stable responsibilities, interfaces, evidence, and acceptance criteria. Then select components that can be operated and sustained in the approved environment.
This is where implementation discipline matters. Aforward-deployed engineering approach for defense AIcan connect mission users, platform teams, security stakeholders, and evidence in the actual delivery environment. Decision Terrain’sair-gapped agentic AI implementation servicefocuses on that integration work while preserving the customer’s model and platform choices.
If you are evaluating Decision Terrain as the platform in this architecture, review DT’s air-gapped deployment configurationand its internal API and MCP integration options. Map the selected models, tools, identity, and execution services to your boundary, then validate the intended workflow against those choices.
Frequently asked questions
Does air-gapped mean the AI system is secure?
No. Isolation changes the threat and dependency profile, but internal identities, software supply chains, removable media, misconfiguration, excessive permissions, data handling, and insider risk still require controls. Security must be designed and assessed for the complete system.
Can an air-gapped architecture use any inference model?
It can support multiple locally deployable models when licensing, hardware, runtime compatibility, security review, and evaluation permit. A stable internal inference interface helps preserve choice, but models are not interchangeable without regression testing.
Should an agent have direct access to mission systems?
Direct broad access is a poor default. Expose only the required actions through narrow, authenticated tool adapters. Begin read-only where possible, apply least privilege, require human approval for consequential actions, and retain evidence of each invocation.
Is this an official DoD reference architecture?
No. This is an independent educational reference informed by public NIST and DoD guidance. It is not an authorization decision, a control baseline, or an architecture issued or endorsed by DoD, NIST, CISA, or any intelligence agency.