On-premise vs cloud AI for sensitive documents: risk, cost, compliance

The on-premise versus cloud question for AI on sensitive documents tends to be argued at the wrong altitude. One camp treats it as an aesthetic preference for sovereignty. The other treats it as a procurement convenience question best resolved by the existing cloud relationship. Neither lens captures what the decision actually involves.

The honest answer is a specific calculation. It has three inputs: a Transfer Impact Assessment under Schrems II^[1], an incident-exposure analysis under NIS2^[3], and a cost analysis that includes the auditability of whatever processor handles the data. This piece walks through each in turn, then proposes a deployment-pattern decision tree that holds up across legal, healthcare, finance, and public-sector work.

The Schrems II calculus

The 2020 Schrems II judgment did not invalidate cloud transfers from the EU. It invalidated the EU-US Privacy Shield, reaffirmed Standard Contractual Clauses (SCCs), and imposed a new obligation on every transfer: the exporter must conduct a Transfer Impact Assessment establishing that the law in the recipient country provides an essentially equivalent level of protection under EU law, or specify the supplementary measures that fill the gap.^[1]

The EDPB's Recommendations 01/2020 set out what supplementary measures look like in practice.^[2] The recommendations distinguish three classes:

1. Use Case 1: encrypted data with keys held in the EU. Acceptable, because the foreign processor cannot meaningfully decrypt.
2. Use Case 2: pseudonymised data where the re-identification key stays in the EU. Acceptable under similar conditions.
3. Use Case 3: cleartext access by the processor. The recommendations are explicit: no contractual or organisational measure can compensate for cleartext access by a processor subject to disproportionate surveillance powers. The transfer fails the assessment.

The 2023 €1.2 billion fine against Meta for EU-US transfers^[4] shows the assessment is enforced, not theoretical. It is the largest GDPR fine on record. The pattern is enforceable not because Meta is special, but because the transfer architecture failed Use Case 3 and the supervisor said so.

Where this lands for AI on sensitive documents

Cloud-hosted AI for sensitive documents typically means cleartext access. The model needs to read the document to process it. If the model is hosted by a US provider, your Transfer Impact Assessment has to account for the US legal regime (FISA 702, Executive Order 12333, the CLOUD Act). The EDPB's Use Case 3 analysis applies. The supplementary measures most cloud AI vendors offer (regional data centers, contractual restrictions, "no government access" claims) do not, on the EDPB's reading, override the underlying statutory exposure.

Three paths exit this problem:

• EU-based provider that owns the entire stack (model, hosting, key management). Transfer Impact Assessment may pass; the residual analysis is on local enforcement, not cross-border surveillance.
• On-premise or sovereign cloud deployment where the model runs in your environment and the data never crosses a jurisdictional boundary.
• Strong supplementary measures, principally client-side encryption with EU-held keys, that put the deployment into Use Case 1 of the EDPB recommendations.

What does not exit the problem: SCCs alone, regional data residency labelling alone, vendor contractual assurances alone.

The NIS2 incident-exposure calculation

NIS2, the EU's expanded cybersecurity directive, took national-transposition effect across the EU in late 2024. Its scope covers a long list of essential and important entities including healthcare, public administration, banking, and digital infrastructure providers.^[3]

For our purposes, two articles do the work.

Article 21 risk-management measures

Article 21 requires essential and important entities to implement appropriate technical, operational, and organisational measures to manage cybersecurity risks. The list is detailed: risk analysis, incident handling, business continuity, supply-chain security, vulnerability handling, access control, encryption, multi-factor authentication, training, and crucially: assessing the cybersecurity practices of suppliers and direct service providers.

Cloud AI providers are direct service providers. The Article 21 obligation lands on you, not on them. Their security posture, contractual flow-down, ability to support your incident response, and observed behavior in past incidents all become part of your compliance posture.

Article 23 incident reporting

Article 23 sets a tight clock on incident reporting. Significant incidents require an early warning within 24 hours, an initial assessment within 72 hours, and a final report within one month. The clock starts when you become aware of the incident.

The operational consequence: a cloud AI provider that takes 48 hours to confirm an incident has occurred in their environment puts you in breach of your 24-hour notification window, even if you respond perfectly. The vendor's incident-response speed is your problem, not theirs.

On-premise deployments do not eliminate this exposure; they relocate it to your own incident-response team. The math is whether your team's clock is shorter than the vendor's. For most regulated organisations that have invested in security operations, the answer is yes.

Norwegian specifics: Sikkerhetsloven

For organisations operating in Norway, an additional layer applies. The Norwegian Security Act (Sikkerhetsloven)^[5] imposes specific requirements on classified information and on entities that handle sensitive information about state security, including supply-chain due diligence and location of processing. Cloud AI processing of materials covered by the Act is, in practice, restricted; the safer pattern is local deployment with documented controls.

For organisations that might end up handling Sikkerhetsloven-touched material (defense suppliers, certain energy and telecoms, parts of the public sector), the deployment pattern often has to be future-proofed even if today's workload is not classified. Designing the architecture for on-premise from the start is cheaper than retrofitting it after a contract win.

The actual cost model

The conventional wisdom is that on-premise AI is more expensive than cloud AI. This is partly true at small scale and partly false at meaningful scale, and the discussion usually omits four cost lines that move the calculation.

Egress and inference unit cost

At scale, cloud AI inference billed per token gets expensive fast. A self-hosted open-weight model on appropriately specified GPUs has a different cost curve: high fixed cost, near-zero marginal cost per request. The crossover depends on volume and model size; for organisations processing tens of thousands of documents per month, on-premise is often cheaper per unit even before compliance considerations.

Audit and TIA costs

Every cloud AI vendor relationship that touches personal data needs a Transfer Impact Assessment and periodic review. Each TIA costs lawyer time, technical review time, and documentation effort. Multiply across vendors. On-premise removes the TIA exposure for that workload entirely.

Incident-response cost

The probability of a cloud AI vendor being involved in a security incident in any given year is non-trivial. Each such incident triggers internal investigation, notifications, customer communication, and often regulator engagement, regardless of whether your specific data was affected. On-premise removes this exposure for that workload.

Vendor lock-in cost

Cloud AI APIs evolve. Pricing changes. Model versions deprecate. A workload that depends on a specific vendor's specific model is exposed to all of those. Open-weight models on on-premise infrastructure trade some peak quality for stability of cost and behavior over time.

A decision tree that holds up

Across regulated work, four questions resolve most deployment decisions.

1. Does the data leave your jurisdiction at any cleartext point?

If yes, the Schrems II analysis applies. If you cannot complete it credibly, the answer is on-premise or sovereign-cloud.

2. Are you a NIS2 essential or important entity?

If yes, your supplier risk-management and incident-reporting clocks include any cloud AI vendor in the pipeline. Vendors who cannot demonstrate 24-hour incident response are not viable. For high-stakes workloads, on-premise reduces the supplier surface.

3. Is the workload Sikkerhetsloven-touched or sector-classified?

If yes or even maybe, on-premise. The cost of retrofitting later is high.

4. Is the volume high enough that per-token billing dominates the TCO?

If yes, on-premise is often cheaper independent of compliance arguments.

If all four answers are no (low-volume, no cross-border issues, not NIS2 in scope, not sensitive sector), cloud AI is genuinely the right call. Most regulated work is not in that quadrant.

A pattern that actually works

The deployment pattern that is most defensible across the four-question screen is:

• Client-side encryption of documents before they leave the user's browser or workstation.
• Local AI processing inside the customer's network, or in a sovereign-cloud region with EU-only operations and EU-resident key management.
• No persistence of cleartext in any external system after processing completes.
• Detailed access logs that survive an audit by the supervisor of any framework in scope.

This is not a coincidence. It is the architecture that simultaneously passes the Schrems II Use Case 1, satisfies NIS2 Article 21 supplier requirements, fits inside Sikkerhetsloven constraints where they apply, and gives the buyer a documentation trail. The fact that it also tends to be cheaper at scale is a bonus, not the point.

The point is that the on-premise vs cloud question for sensitive document AI is not a preference. It is a calculation. Done honestly, the calculation tends to one answer for high-stakes regulated work and a different answer for low-stakes commodity work. The error is treating both kinds of work the same way.