Choosing Between Cloud and In-House AI Infrastructure Based on Security and Scalability Requirements
How data sensitivity class and workload profile shape the choice between cloud and in-house AI infrastructure: threat model, scaling, resilience, regulation.
Oleksandr Ivanchuk
Software Developer
The question of "cloud or own servers" for artificial intelligence systems is usually discussed in terms of cost. In practice, however, the decision is determined by two other variables: how sensitive the data handled by the system is, and how predictable the workload is. In this article we propose viewing the choice of infrastructure through precisely these two axes, security and scalability, and show why for most enterprises the answer is not a single model but a distribution of workloads among several.
Introduction
We have already examined the general framework for choosing a deployment model, including latency, regulatory requirements, and total cost of ownership, in the article "Local AI Deployment vs Cloud". This article focuses on a narrower but most contentious aspect: how exactly to turn security and scalability requirements, which often pull in opposite directions, into a concrete architectural decision.
The contradiction between them is real. Security requirements push toward maximum control: own hardware, own perimeter, no external dependencies. Scalability requirements push in the opposite direction: elastic resources that can be obtained in minutes and returned when the need has passed. An enterprise that optimizes only one of these axes usually ends up with either a secure system that cannot withstand growth, or a flexible system that the security department does not admit to real data.
The terms should also be clarified. By cloud infrastructure we mean resources provided on demand from a provider's shared pool [1]. By in-house infrastructure we mean hardware that the enterprise controls itself: in its own server room, in a leased data center, or in a dedicated private environment. Between these poles lies a spectrum of intermediate options, and it is within this spectrum that most practical decisions are found.
The Security Axis: What Exactly Needs to Be Protected
The conversation about the security of AI infrastructure often comes down to the question "can the cloud provider be trusted." In our view, this is an imprecise formulation. It is more productive to ask: which assets does the system process, and against which threats should they be protected.
In AI systems there are at least four such assets:
- Data for training and fine-tuning: historical production indicators, document archives, labeled examples.
- Query data: what users and systems pass to the model during operation; this is often what contains the most sensitive current information.
- Model parameters: if a model has been fine-tuned on proprietary data, it itself becomes a carrier of trade secrets.
- Agent credentials and access rights: the keys to ERP, MES, and SCADA that the system uses on behalf of the enterprise.
The list of threats to applications based on large language models maintained by the OWASP community shows that a significant share of the risks (injection of malicious instructions into a prompt, disclosure of sensitive information, supply chain vulnerabilities, excessive agent privileges) does not depend on where the computation is physically located [6]. An own server does not protect against an agent that has been granted too many rights, and the cloud does not make a system more vulnerable to a malicious document in the knowledge base.
Thus, the hosting location affects primarily three groups of risks: third-party access to data (including at the request of a foreign jurisdiction), vendor dependence, and physical resilience. The rest is determined by the architecture of the system itself, and this body of work is the same for both models.
I consider it important to note one more circumstance that is often underestimated: in-house infrastructure is more secure than cloud infrastructure only when the enterprise is capable of protecting it at an appropriate level. Updates, network segmentation, access control, and round-the-clock incident response all become its own responsibility. For an enterprise without an established information security function, "own" may mean "less protected."
Data Sensitivity Classes and Permissible Models
In our practice, the decision begins with the classification not of systems but of data. One and the same AI system may work with information of different classes, and the requirements are determined by the most sensitive of them.
| Data class | Examples | Permissible infrastructure | Key conditions |
|---|---|---|---|
| Open and publicly available | Public documentation, catalogs, general reference materials | Any, including public cloud services | Standard contractual terms |
| Internal | Regulations, instructions, standard reports | Cloud with contractual guarantees that data is not used for training | Access control, logging |
| Confidential | Formulations, process parameters, commercial terms, personal data | Dedicated cloud resources in the required jurisdiction or in-house infrastructure | Encryption, key control on the enterprise side, data transfer impact assessment |
| Critical | Control system data, restricted-access information, data of critical infrastructure facilities | In-house isolated infrastructure | Physical control, no external dependencies within the perimeter |
Regulatory requirements set the lower bound here. The NIS2 Directive establishes cybersecurity risk management obligations, including in the supply chain, for a wide range of sectors, manufacturing among them [3]. The EU Artificial Intelligence Act requires high-risk systems to have an appropriate level of accuracy, robustness, and cybersecurity [2]. In Ukraine, relations concerning cloud services, including their use by state bodies and critical infrastructure operators, are regulated by a separate law [8]. None of these documents prohibits the cloud as such, but all of them require that the enterprise be able to demonstrate control over where and how data is processed.
Useful reference points for building such control are the AI Risk Management Framework developed by NIST [4] and the ISO/IEC 42001 standard, which describes an artificial intelligence management system [5]. Both documents are applicable regardless of the deployment model.
The Scalability Axis: What Workload Must Be Sustained
The second axis concerns not the volume of the workload but its nature. From an infrastructure standpoint, AI workloads fall into several profiles with fundamentally different requirements.
Steady and predictable. A quality control system on a line that runs around the clock; regular processing of document flow. Hardware utilization is high and constant. Here in-house infrastructure is used efficiently, and the cloud pay-per-use model loses its main advantage. A Lenovo study gives an estimate that for high-utilization workloads own hardware pays for itself in less than four months; this figure should be treated as a manufacturer's estimate, yet it does reflect the general pattern [7].
Peak and irregular. Periodic retraining of models, quarterly analytics, seasonal surges. Hardware purchased for the peak stands idle most of the time. This is the natural niche of the cloud.
Growing with an uncertain trajectory. Pilot projects and new scenarios where it is unknown whether the workload in a year will be ten times larger or zero. Capital investment under such uncertainty is the riskiest option.
Geographically distributed. Several sites, each with its own response time requirements. Here edge computing in combination with a central node comes to the fore.
It should be taken into account separately that scaling in-house infrastructure is not instantaneous: procuring specialized hardware and preparing power supply and cooling take months. The scalability of an in-house environment is a matter of planning, not of operational response.
The matrix below summarizes our assessment of the suitability of the three models for typical classes of workloads.
Resilience as a Separate Dimension
For Ukrainian enterprises, a third axis is added to the two named above, one that European overviews usually mention only in passing: physical resilience. An own server room on the enterprise's premises depends on the local power supply, communication links, and the physical integrity of the building. The experience of recent years has shown that for a significant share of organizations, moving systems and backups to geographically distributed data centers was a decision in favor of continuity, not one made at the expense of security.
This changes the customary logic. The argument that "own means more reliable" holds only where there is backup power, redundant communication links, and a remote backup site. If these conditions are absent, in-house infrastructure is a single point of failure. In our practice, even enterprises with strict confidentiality requirements increasingly choose an arrangement in which the working environment remains in-house, while encrypted backups and reserve capacity are located off-site.
How to Combine the Two Axes in a Single Decision
If the data classification is overlaid on the workload profile, most situations are resolved without conflict.
- Low sensitivity, any workload: cloud. In-house infrastructure offers no advantages here that would justify the costs.
- High sensitivity, steady workload: in-house infrastructure. Both the security requirements and the economics coincide.
- High sensitivity, peak workload: the real zone of conflict. Typical solutions: dedicated cloud resources in the required jurisdiction with encryption keys under the enterprise's control; anonymization or aggregation of data before it is transferred outside; moving peak tasks to less sensitive derived data.
- Mixed sensitivity: distribution at the level of individual tasks: sensitive steps are executed in the in-house environment, the rest in the cloud. Such a distribution is especially natural for agentic systems, where different agents have different privileges.
The last option deserves particular attention. The hybrid model is not a compromise but a way to apply to each workload the infrastructure that suits it. However, it has its own price: two environments must be protected, observed, and kept consistent, and the boundary between them itself becomes a target of attack. We examined the question of choosing models for each of the environments in the article on open and commercial AI models.
Author's Assessment and Conclusions
The choice of infrastructure begins with data classification, not with a comparison of providers. Until the enterprise has determined which data belongs to which class, any decision will be either excessively cautious or unjustifiably risky.
The hosting location solves only part of the security tasks. Most AI-specific threats are related to the application architecture and agent privileges and demand equal attention in both models [6].
In-house infrastructure is an obligation, not only an asset. It provides control to the extent that the enterprise has the competencies and resources to exercise that control. An honest assessment of its own cybersecurity capabilities must precede the decision.
Scalability should be assessed by the nature of the workload. A steady workload justifies own hardware; a peak or uncertain one justifies the cloud. Buying capacity for a hypothesis that has not yet been confirmed is the most expensive mistake.
Resilience must be considered separately from confidentiality. For Ukrainian enterprises, the geographic distribution of backups and capacity is as much a security requirement as protection against unauthorized access.
The decision is not final. A scenario that began as a cloud pilot may, once its value is confirmed, be moved into the in-house environment, and vice versa. An architecture that permits such a move without rebuilding the system is more valuable than the optimal choice for today.
In our assessment, over the coming years the standard for medium and large industrial enterprises will become a distributed model in which the place of execution of each task is determined by the data class and the workload profile. The task of the executive is not to pick a side in the "cloud versus own servers" dispute, but to ensure that the enterprise has rules by which this decision is made for every new system.
References
- NIST. "The NIST Definition of Cloud Computing — Special Publication 800-145." 2011. https://csrc.nist.gov/pubs/sp/800/145/final
- European Parliament and Council. "Regulation (EU) 2024/1689 (Artificial Intelligence Act)." 2024. https://eur-lex.europa.eu/eli/reg/2024/1689/oj
- European Parliament and Council. "Directive (EU) 2022/2555 (NIS 2 Directive)." 2022. https://eur-lex.europa.eu/eli/dir/2022/2555/oj
- NIST. "Artificial Intelligence Risk Management Framework (AI RMF 1.0)." 2023. https://www.nist.gov/itl/ai-risk-management-framework
- ISO/IEC 42001:2023. "Information technology — Artificial intelligence — Management system." https://www.iso.org/standard/42001
- OWASP. "Top 10 for LLM Applications." 2025. https://genai.owasp.org/llm-top-10/
- Lenovo. "On-Premise vs Cloud: Generative AI Total Cost of Ownership (2026 Edition)." 2026. https://lenovopress.lenovo.com/lp2368-on-premise-vs-cloud-generative-ai-total-cost-of-ownership-2026-edition
- Verkhovna Rada of Ukraine. Law of Ukraine "On Cloud Services" No. 2075-IX. 2022. https://zakon.rada.gov.ua/laws/show/2075-20