An air-gapped AI deployment runs language models, retrieval, and the applications built on them entirely on hardware that has no connection to the internet or to any outside network. Models and updates arrive on approved physical media, nothing phones home, and the organisation carries every operational task a cloud vendor would otherwise handle.
What “air-gapped” actually means, and what it does not
An air gap is a physical separation: the machines running the AI have no network path to the outside world - no internet, no VPN, no vendor tunnel, no cloud API. Data can only get in or out on removable media that passes through a controlled transfer process. That is stricter than on-premise AI in the usual sense, where servers sit in your own building but can still reach out for model updates, package installs, or licence checks. Plenty of organisations that say “air-gapped” actually mean “on-premise with a strict firewall.” The difference matters, because a true air gapped AI deployment changes how every routine task is done, from installing a library to reading a log.
Who actually needs it
Defence and intelligence environments, classified government networks, and some critical-infrastructure operators are required to run this way. Beyond those, it shows up in a few places by choice: research labs protecting unpublished results, law firms and banks handling matters where even metadata leaving the building is a problem, manufacturers with proprietary process data, and organisations in jurisdictions with hard data-residency rules. If your requirement is really “our data must not go to a third-party AI vendor,” a connected on-premise deployment usually meets it at far less operational cost - the companion post on on-premise AI for government agencies walks through that middle ground. Choose the air gap when a policy, a contract, or a regulator requires it, not because it sounds safer.
What runs inside the gap
Open-weight models you can carry in on media
The models have to be ones whose weights you can legally download and run yourself - the open-weight families from Meta, Mistral, Alibaba’s Qwen, DeepSeek, and others. Proprietary models served only through an API are simply not an option, whatever their quality. Model choice is driven by the task, the hardware you can put in the room, and the licence terms, which you should read before, not after, the model is copied. Our guide to running Llama on-premise covers the model-selection questions in more depth.
Inference serving, retrieval and the application layer
A typical stack is an inference server such as vLLM or Ollama, a local vector database for retrieval over your documents, an embedding model, and whatever application sits on top - an internal assistant, a document search tool, or an AI agent that acts inside your own systems. Every one of these components, with every dependency, has to be brought in as a complete, verified bundle. There is no “pip install” from inside the gap.
Hardware sized for the workload, not the demo
GPU servers do the inference; how many and which class depends on the model size, how many people use it at once, and how fast responses need to be. Inside an air gap you cannot burst to the cloud when the room is busy, so sizing has to account for peak use plus headroom for the models you will want next year. Spare parts and a replacement plan matter too: a failed card cannot be swapped for a cloud instance while you wait for shipping.
Required to run AI with no outside connection?
Tell us the environment and the workload and we’ll tell you plainly what an air-gapped deployment would take - and whether a connected on-premise build would meet the requirement instead.
Talk to an On-Premise AI SpecialistHow a deployment actually happens
The build is done outside the gap first. The team assembles the full software bundle - models, serving stack, retrieval components, application code, and every dependency - on a connected staging environment that mirrors the target hardware, tests it end to end, and records checksums for every file. The bundle is written to approved media, moves through the organisation’s transfer process, and is verified against those checksums on the inside. Only then is it installed, configured, and tested again against the real documents and systems. The same path is used for every later update, which is why the first deployment should also establish the update routine rather than treat it as a one-off.
What breaks when you cut the cable
More than most teams expect. Software that quietly checks for updates or reports usage hangs or fails. Package managers try to reach a registry. Container images assume they can pull base layers. Model-serving frameworks may try to download tokenizer files or configuration on first run. Licence checks phone home. Time synchronisation, certificate validation, and DNS all behave differently when there is nowhere to reach. A team that has only run the stack in a connected lab discovers these one at a time, on site, under time pressure. The staging environment should therefore be disconnected too, so that every one of these failures is found before the real move.
Keeping it running: updates, patches and monitoring
Every model update, security patch, and application change follows the same media-transfer path, so update cadence becomes a deliberate schedule rather than a background process. Vulnerability management needs a way to know what is installed inside the gap and compare it to advisories outside it. Monitoring and logging stay inside: dashboards, alerts, and audit logs are read by people in the room or exported outward through the same controlled process, never streamed to a vendor. Someone has to own all of this. In practice that is either an internal platform team with the skills, or a partner who builds the environment, documents the runbook, and trains that team - the model we use in AI agent development engagements that end behind an air gap.
Security is still your job inside the gap
An air gap stops network attacks from outside. It does nothing about a compromised file carried in on media, an insider with access to the console, or a model prompted into leaking a document it should not have retrieved. Access control on the machines, role-based permissions in the application, retrieval that respects document-level access rights, and review of what goes onto transfer media all still apply. Treat the air gap as one control among several rather than the control that makes the others unnecessary.
Cost and effort, without a fake number
We do not quote prices in guides, and any vendor who gives you a figure before understanding the workload is guessing. What we can say is where the cost sits: in the hardware, which you buy outright and size for peak; in the initial build, which is longer than a connected deployment because every component is bundled, verified, and tested twice; and in ongoing operations, because a team is doing by hand what cloud vendors automate. The offset is that there is no per-token or per-seat bill, no data-processing agreement with an AI vendor, and no exposure to a provider changing terms. The self-hosted LLM for business guide covers the cost drivers in the connected case, and most of them apply here with extra operational weight.
Questions to ask a vendor before committing
Ask whether they have delivered a truly disconnected environment before, and what broke. Ask how the update routine works and who runs it after handover. Ask which model licences they have checked for your intended use. Ask how retrieval enforces document permissions so the assistant cannot surface a file to someone who could not open it directly. Ask what the runbook covers when a GPU fails or a model needs rolling back. Ask for a reference from an organisation with similar constraints - and expect that reference to be limited in what it can say.
Deciding between air-gapped and connected on-premise
If a regulation, a classification level, or a signed contract requires no outside connectivity, the decision is made for you and the job is to do it well. If the requirement is that data must not leave your control, a connected on-premise deployment behind your own firewall usually satisfies it, keeps automated updates and vendor support available, and costs considerably less to operate. Write the actual requirement down in one sentence, check it with whoever enforces it, and choose the architecture that meets that sentence rather than the one that sounds most secure.
Getting started
Start with one contained use case - most often document search or an internal assistant over a fixed set of material - and one model that fits the hardware you can put in the room. Build and break it on a disconnected staging environment first. Establish the transfer and update routine as part of the first deployment, not afterwards. Inwizards builds and supports these environments from teams in the US, UAE and India; the on-premise AI page explains what a deployment includes and how an engagement runs.