Self-hosting or a cloud platform for AI agents? Deployment selection and TCO evaluation checklist
When choosing how to deploy AI agents, the easiest mistake to make is comparing only a GPU bill with a cloud service quote. The outcome is also affected by model services, orchestration, skills, memory, logs, permissions, upgrades, and incident handling. To decide between self-hosting and a cloud platform, compare the complete total cost of ownership (TCO) and control boundaries.
First, confirm which components you actually need to deploy
- Models or external model APIs: model versions, inference resources, quotas, and fallbacks.
- Agent orchestration: task states, serial and parallel dependencies, timeouts, and retries.
- Skills and integrations: files, databases, business APIs, and third-party tools.
- Memory and knowledge: cross-session information, business documents, retrieval, and permission isolation.
- Operations and governance: logging, monitoring, auditing, secret management, backups, and upgrades.
Self-hosting means that the team is responsible for the availability and security of these components, while a cloud platform delegates some of that responsibility to the provider. The difference is not whether the components can be installed, but who assumes responsibility for them.
A quick comparison across six dimensions
| Dimension | When self-hosting has an advantage | When a cloud platform has an advantage |
|---|---|---|
| Data and compliance | Data must remain within designated network or storage boundaries | Managed services may be used under clearly defined agreements and permissions |
| Team capabilities | Model, platform, security, and operations personnel are already available | The business team wants to validate the value of the process first |
| Time to launch | A longer implementation and validation cycle is acceptable | Piloting and iteration need to begin as soon as possible |
| Depth of customization | Underlying components or special network topologies need to be modified | Standard APIs, skills, and workflows already cover the requirements |
| Workload characteristics | The workload is stable over the long term, and resource utilization is controllable | Traffic fluctuates significantly or demand remains uncertain |
| Responsibility boundaries | The team is willing to handle upgrades, monitoring, and incident recovery itself | The team wants the provider to handle infrastructure operations |
What should be included in TCO?
At a minimum, divide costs into four groups: one-time implementation, ongoing operation, operations personnel, and change-related risks. Hardware procurement is only one item.
- Implementation: architecture design, environment setup, security reviews, system integration, and migration.
- Operation: compute, storage, networking, model calls, and third-party APIs.
- Personnel: on-call coverage, upgrades, capacity planning, vulnerability remediation, and troubleshooting.
- Risks: scaling delays, version compatibility, downtime, vendor lock-in, and exit costs.
Comparing only the price per call will usually underestimate the labor and idle resources associated with self-hosting, while potentially overlooking the cloud platform call charges that accumulate as usage grows. Compare the options based on the same business volume, quality targets, and availability targets.
When should self-hosting be prioritized?
- Data cannot leave a controlled environment, and clear security and audit requirements are already in place.
- Existing internal systems are highly customized, and standard integration methods cannot meet the requirements.
- The team has the ability to continuously maintain model services, orchestration, storage, and monitoring.
- The workload is relatively stable, allowing resource utilization to be evaluated using real data.
When should a cloud platform be prioritized?
- The business value has not yet been validated, and a low-risk pilot needs to be completed first.
- There is no dedicated AI infrastructure team.
- Workloads fluctuate significantly, making on-demand scaling desirable.
- The focus is on workflows, skills, and delivery rather than maintaining underlying components.
A compromise: divide the deployment by data and task
Deployment does not have to be an either-or choice. Sensitive data processing can remain in the internal environment, while general-purpose model capabilities or burst workloads use cloud services. Alternatively, a cloud platform can first be used to validate inputs, outputs, and acceptance criteria before deciding which stable processes are worth migrating. A hybrid approach requires clearly documented data flows, credentials, and audit boundaries.
Replace theoretical debate with a pilot
- Select a low-risk process with stable inputs and results that can be reviewed manually.
- Record setup time, call costs, manual intervention, causes of failure, and maintenance work for each option.
- Apply least-privilege access to data, and prepare revocation and cleanup plans.
- After the trial run, review the results based on TCO, quality, speed, and governance requirements.
- Confirm the exit process: how data will be exported, how skills will be migrated, and how credentials will be revoked.
Frequently asked questions
Is a cloud platform always cheaper?
Not necessarily. When requirements are uncertain or workloads fluctuate, a cloud platform usually reduces upfront investment. For large-scale workloads that remain stable over the long term, self-hosting may be more worthwhile to evaluate. The conclusion must be based on the team’s actual usage and labor costs.
Is self-hosting always more secure?
Not necessarily. Greater physical control does not automatically establish effective governance. Permissions, patches, secrets, logs, and backups still require continuous management.
What should you focus on when evaluating SmaugBrain?
Focus on which workflows, skills, and collaboration requirements it can support, and whether its data handling, permissions, logging, models, and exit mechanisms meet the team’s requirements. Do not rely solely on the feature list; validate it through a pilot.