Private AI agents: why regulated firms run them in their own environment.
A quiet shift in this autumn’s agent announcements: more of them run in the customer’s environment. Self-hosted versions of coding agents, managed runtimes with isolated sandboxes, hardware for running agents locally. For healthcare, legal and financial firms, that is not a feature — it is the condition for saying yes.
The agent reads patient, client or privileged data. Its actions touch systems of record. You need to prove to an auditor where data went. Any one of these is enough.
What “in your environment” should mean
The agent’s code, prompts, evaluations and logs run in your cloud account. Model calls go through endpoints with no-training, no-retention terms — or to a model you host yourself. Credentials stay in your vault. And you can switch it off without asking anyone.
What it costs, and what you give up
Running inside adds setup — networking, identity, monitoring — which is part of why a production agent costs more than a chat widget. You also take on some operations, unless you contract them out. What you gain is the ability to answer “where did that data go?” in one sentence.
The vendor questions
Can it run in our account? Who holds the keys? Which model providers see our data, and under what terms? Can we export everything — code, prompts, evaluations — if we part ways? If the answer to the last one is no, you are renting, not owning.
How we do it
Every production agent we build ships in the client’s cloud, under the client’s accounts, with the repository in the client’s name. That is in our contracts, not just our pitch.
A worked example: a law firm’s research agent
A mid-size firm wants an agent that searches its own memo bank and research subscriptions. Run as a hosted service, the firm’s memos would sit in someone else’s infrastructure. Run in the firm’s own cloud account, the memos never leave; the agent calls a model endpoint under no-training, no-retention terms; and access follows the firm’s existing permissions, so a trainee cannot surface a partner-only matter. Same agent, very different answer to the general counsel’s first question.
The trade-offs, plainly
You gain control of data, audit trails you own, and the ability to leave a vendor without losing your work. You give up some convenience: setup takes longer, and someone has to watch costs, uptime and updates. For regulated firms the trade is usually worth it; for a marketing chatbot, often not.
Common mistakes
Assuming “private” means local models only. Many firms use frontier models through enterprise endpoints with strict data terms — the privacy comes from the contract and architecture, not just the model’s location.
Forgetting the logs. Prompts and outputs are data too. Keep them in your environment, with retention you set.
Our free AI vendor scorecard grades any vendor — us included — in three minutes. Related: do you own the AI code?
Frequently asked questions
What is a private or self-hosted AI agent?
An agent whose code, data, logs and credentials run inside your own cloud account or infrastructure, so you control where data goes and can switch it off yourself.
Do private agents have to use open-source models?
No. They can call commercial models through enterprise endpoints with no-training and no-retention terms, or use models you host — whichever meets your risk and quality needs.
Is a self-hosted agent more expensive?
Setup is higher because of networking, identity and monitoring, and someone must operate it. For regulated data the extra cost usually buys compliance you would otherwise struggle to prove.
Source: AI Agents News Brief, October 2026.