Private AI on Your Own AWS or Azure Account

Honest Consultancy. Updated 3 November 2026

What does private AI actually mean?

Your application calls a model through a service in your own AWS or Azure account, under your cloud agreement, and your documents stay in storage you control. The two main services are Amazon Bedrock and Microsoft Foundry, which offers Azure OpenAI models among others.¹²

That is different from staff pasting text into a public chat tool, where the terms depend on the plan they signed up for and you may have no logs at all. It is also different from running a model on your own servers. In a managed service the provider still operates the hardware and the model, and you are relying on its commitments, so read them.

Private does not mean your cloud provider never sees anything. Both providers describe abuse monitoring, which can include storing and reviewing some content in defined circumstances. The details are below.

Do the model makers see our prompts?

On these services, the providers say no. AWS says model providers have no access to the Bedrock deployment accounts, so they have no access to customer prompts and completions or Bedrock logs.¹ Microsoft says prompts, outputs, embeddings and training data are not available to OpenAI or other model providers.²

Microsoft also says this data is not used to train generative AI foundation models without your permission or instruction, and that its hosted models do not interact with services run by the model makers, such as ChatGPT or the OpenAI API.²

Check the current terms for the specific model you pick, and record the date you checked. On Bedrock, some models carry their own requirements, covered next.

Are prompts stored anywhere?

It depends on the model and your settings, so check instead of assuming. Amazon Bedrock lets you configure data retention by Region, at account or project level, and a Region you have not configured falls back to each model's default.³

Features you switch on can store data too. On Azure, the Responses API, Threads and stored completions keep message history in your resource, encrypted at rest, and you can delete it.² Decide whether you want them, and set a retention period that matches your privacy notice.

With zero data retention set, AWS says no request or response data is written to durable storage or shared with the model provider, and a model that requires retention is blocked.³
Under the default mode, AWS may retain data for safety and abuse prevention, and the model provider does not receive it.³
Some models require human review as a condition of access. For those, AWS says prompts and completions are retained within AWS for up to 30 days and may be reviewed by AWS, and are not shared with the provider.³
On Azure, a sample of prompts and completions may be selected for review if abuse monitoring flags them. For models deployed in the EEA, Microsoft says its human reviewers are located in the EEA.²
Eligible Azure customers can apply for modified abuse monitoring, which stops that storage and human review, while automated review may continue.²

Where is the data processed?

Do not assume it stays in the Region you selected. This is the setting to check first, because both providers offer modes that process requests elsewhere in exchange for capacity.

AWS advises checking the destination Regions for every inference profile you plan to use, rather than relying on the prefix in its name.⁴ It also says that where it stores data for abuse detection, that data sits in the destination Region.⁴

For Irish businesses the question is often less about Ireland and more about the EU. Moving between EU regions may be perfectly acceptable to you and your customers, whereas a global route may not be. Write the decision down, and check it against your contracts and privacy notice. New models are often released to certain Regions first, so your preferred model may not be offered in Ireland on the day you need it. Check the regional availability lists.⁵

Azure standard deployments process prompts within your chosen geography, though possibly between regions in it.²
Azure DataZone deployments in an EU member state may process prompts in that or any other EU member state.²
Azure Global deployments may process prompts in any geography where the model is deployed, although data at rest stays in your geography.²
On Bedrock, geographic cross-Region inference keeps requests within a geography such as the EU, though prompts and outputs may move outside your source Region.⁴
Global cross-Region profiles on Bedrock can route requests worldwide.⁴

How do we keep the traffic private?

Use private network paths and the access controls you already have. AWS supports Amazon VPC and AWS PrivateLink for Bedrock, and Azure supports network controls for its service.⁶⁷

Reach the model through a private endpoint, so traffic does not cross the public internet.
Give each application its own identity and the narrowest permissions it needs.
Encrypt stored data with keys you control where the service supports them. Microsoft offers customer managed keys, with some preview features excepted.²
Turn on logging. AWS recommends CloudTrail for API and user activity on Bedrock.¹
Do not put personal data in names or tags. AWS warns these may be used in billing or diagnostic logs.¹

How do we let it answer from our own documents?

With a retrieval layer: the application finds the relevant passages in your documents and sends only those to the model with the question. On Azure, Microsoft says the data stays in the source you designate and that the service does not create a duplicate store.²

The common mistake is access. If an assistant can search every file in the company, any user can reach any file by asking the right question, including HR and legal folders. Build retrieval so it respects each user's existing permissions, and start with a small set of documents that everyone may read.

Quality comes down to the documents too. Old policies, duplicates and conflicting versions lead to confident wrong answers, so tidying the source material is often the most valuable work on the project. Test with real questions and check the answers against the source, and have the assistant cite which document it used so people can check it themselves.

Does running it privately make us GDPR compliant?

No. It helps with some of the questions the Data Protection Commission asks, such as where data goes and whether a provider re-uses it, but you remain responsible.⁸⁹

You still need a lawful basis and a clear purpose, data processing terms with the provider, a retention period, and a way to meet access and deletion requests.⁹ Our guide to AI agents and GDPR for Irish SMEs goes through them, and our EU data residency guide covers the wider cloud questions.

What does it cost?

Mostly usage, plus the systems around the model. These services are generally charged by volume of text processed, so cost grows with the length of prompts and answers, and with how many people use it. Check current rates on the provider's pricing page before you size a project.¹⁰¹¹

Budget for the parts around the model: the search index, storage, logging, monitoring and the engineering to run it. Set usage budgets and alerts from the first day, because an agent that retries in a loop can spend quickly. Provider usage data will help, so review the costs in the first weeks and pick the smallest model that does the job well enough.

Frequently asked questions

Is Amazon Bedrock or Azure better for private AI?

Start from where your data and skills already are. Both publish similar commitments on model providers, and the model you need may decide it. Our guide to AWS vs Azure covers the wider choice.

Can we keep AI processing in Ireland?

Only with the right deployment type and a model offered in that region. Cross-region modes can process requests elsewhere, so check the destination regions before you promise it.²⁴

Will our data be used to train the models?

AWS and Microsoft both say prompts on these services are not shared with model providers, and Microsoft says they are not used to train foundation models without your permission or instruction.¹²

Do we need to train our own model?

Rarely. Most business uses work well by connecting an existing model to your documents through retrieval, which is simpler and easier to keep up to date.

Sources

See software, data and AI See the AI opportunity review

Private AI on your own AWS or Azure account