Artificial Intelligence on Microsoft Azure
Artificial Intelligence on Microsoft Azure
If your organisation already runs Microsoft 365, Entra ID and SharePoint, several of the hardest parts of an AI programme are in place: identity, permissions, sensitivity labelling and a content estate. Azure lets you build on that rather than beside it, which turns the work into grounding, evaluation and control instead of standing up a parallel access model. We use Microsoft Foundry for custom applications and Copilot Studio or Microsoft 365 Copilot where configuration is genuinely enough.
Why Azure
When this is the right platform.
- Your access model already exists. Microsoft Entra ID decides who sees what, and retrieval through Azure AI Search can honour those same permissions at query time rather than maintaining a second list of who may read which document.
- Microsoft Foundry keeps model deployment, grounding, content filtering, evaluation and quota control in one place, which shortens the distance between a prototype and something you can raise a change record against.
- Copilot Studio and Power Platform cover the last mile. A business team can own the interface and the approval flow while engineering owns retrieval, data contracts and guardrails, without two separate delivery projects.
- Microsoft Fabric and Microsoft Purview put analytical data and governance metadata in the same tenancy as the AI workload, so lineage and classification are not a separate programme bolted on later.
- If your team already deploys with Bicep, GitHub Actions or Azure DevOps and troubleshoots with Application Insights, an AI workload joins the operating model you run today instead of introducing a second one.
Where it is less suited
We would rather say this now than after a migration.
- Microsoft 365 Copilot is charged per user per month whether or not it is used, so the commercial case rests on adoption rather than deployment. We would rather start with a small allocation to well-matched roles and expand on evidence than licence a whole department and hope.
- Grounded answers are only as good as your permissions and your content. A decade of oversharing and duplicate documents in SharePoint becomes an AI problem the moment you switch an assistant on, and repairing it is an information-management project that sits outside the AI build and needs its own budget.
- This part of Azure moves quickly. Naming has shifted more than once (Azure OpenAI, then Azure AI Foundry, now Microsoft Foundry), and model availability differs between regions and does not arrive everywhere at the same time, so runbooks need a refresh cadence and you may face a real choice between the newest capability and the region you prefer.
- Power Platform makes it easy for business teams to build things nobody owns. Without an environment strategy and data-loss prevention policies in place first, an AI programme quietly inherits an estate of unsupported flows and apps.
Business outcomes
What Azure delivers here.
- A knowledge assistant that honours existing permissions
- Staff question your own SharePoint and Teams content and receive cited answers limited to what their Entra ID account already grants them. The measure agreed up front is usually the share of questions resolved without escalating to a subject-matter expert.
- Copilot licences pointed at the roles that benefit
- Instead of a tenant-wide rollout, licences go first to the roles whose daily tasks match what the tool does well, with a review point before any expansion. Finance gets a per-seat cost set against an observed change in a named task rather than a general productivity assertion.
- Document work that finishes inside familiar tools
- Contracts, tenders, invoices and forms are extracted and summarised by a pipeline, with the human review step surfaced in Teams or a Power Apps screen the team already opens each morning. The process owner tracks turnaround against the baseline captured before the build.
- Agents that act, with an approval in front of them
- A Copilot Studio agent can raise the request, update the record or start the workflow in your line-of-business system, with a Power Automate approval ahead of anything consequential. Benefit shows up as manual handoffs removed from a named process.
- Forecasts built next to the governed data
- Azure Machine Learning models draw on the same Microsoft Fabric data that feeds your reporting, so a forecast and a dashboard cannot disagree about the underlying numbers. Planners receive the model error alongside each prediction.
Common client problems
What we usually hear first.
We switched Copilot on and half the team says the answers are shallow.
That is normally a content and permissions problem rather than a model problem, so we start by establishing what the tool can actually reach and how well it is labelled. We then narrow the scope to well-maintained sources, remove the sharing that should never have existed and rebuild enablement around specific tasks.
Our SharePoint is 10 years of unsorted folders. Search is already bad.
We assess the sites in scope for duplication, staleness and oversharing, and we will tell you plainly if information management has to come before an assistant. Where the corpus is only partly usable we scope the assistant to the parts that are trustworthy and declare the rest out of scope in writing.
The business keeps building Power Automate flows and nobody knows what exists.
We inventory the flows and connections in each environment, separate what is business critical from what has been abandoned, and put an environment and data-loss prevention structure around new development. Agents are then built inside that structure instead of adding another undocumented layer.
We were sold Azure OpenAI, then Azure AI Foundry, now Microsoft Foundry. What do we actually have?
We map your existing resources, deployments and commitments to the current product naming and separate what changed in substance from what changed in name only. You get a current-state diagram and a short migration list covering the few things that genuinely need to move.
Legal will not approve anything that sends contract text outside our control.
We scope which repositories are in play, use Purview labelling to keep restricted content out of the index, and configure private endpoints so model and search traffic does not traverse the public internet. Where legal rules a source out entirely, we design the workflow without it rather than argue the point.
How we deliver
Our Azure delivery approach.
- 01
Assessment and advisory
The Azure assessment starts inside your tenant, because most of the answers about feasibility are already sitting there.
- Tenant permission review. Entra ID groups, guest access, sharing links and SharePoint permission inheritance, because these become the AI access model on day one.
- Content hygiene analysis. Duplication, stale documents and oversharing across the libraries in scope, each of which quietly degrades a grounded answer.
- Licence against consumption. Microsoft 365 Copilot, Power Platform and Microsoft Foundry costed per seat and per call for every candidate use case, so the cheaper path is obvious.
- Regional availability check. Model and capability availability confirmed in the Azure regions you intend to use, recorded with the date it was verified because it changes.
- Build or configure ruling. Where Microsoft 365 Copilot or Copilot Studio is already sufficient, and where a custom Foundry application is genuinely justified.
- Purview readiness. Whether sensitivity labels, retention and classification are applied consistently enough for an indexing pipeline to rely on them.
- 02
Architecture and implementation
Three reference patterns cover most of what organisations need on Azure: a grounded assistant, a document-processing pipeline and a controlled agent. Each is built incrementally and deployed through your existing pipeline.
- Hybrid retrieval on Azure AI Search. Semantic ranking, security trimming by Entra ID group, and an indexing schedule matched to how often each source actually changes.
- Model configuration made explicit. Deployment in Microsoft Foundry with content filters set deliberately, per-application quota and rate limits, and Foundry Models compared on your own evaluation set.
- Runtime chosen for fit. Azure Functions for event-driven extraction, Azure App Service for a hosted assistant, Azure Kubernetes Service only where you already operate a cluster.
- One gateway for every call. Azure API Management in front of model and tool endpoints, giving authentication, throttling, per-consumer quota and a single consistent log.
- Managed solutions, never live edits. Copilot Studio agents, Power Automate flows and Power Apps screens promoted through development, test and production rather than authored in production.
- Versioned predictive models. Azure Machine Learning with registered models and versioned datasets, drawing features from Microsoft Fabric so forecasts and reports share one governed source.
- 03
Security and governance
On Azure the controls you need mostly already exist in the tenant. The work is applying them to the AI workload deliberately and proving they hold.
- Managed identities, not keys. Entra ID app registrations scoped to least privilege, secrets removed where a managed identity will do, and conditional access on the assistant front end.
- Trimming proven, not assumed. Retrieval tested with accounts at three permission levels, including one account that should be able to see nothing at all.
- Labels honoured at ingestion. Microsoft Purview sensitivity labels enforced by the pipeline, with the excluded set published rather than left implied.
- Prompt injection treated as hostile input. Retrieved text treated as untrusted, tool allowlists enforced, and adversarial cases added to the evaluation suite alongside Foundry content filters.
- Findings in your existing queue. Microsoft Defender for Cloud across the AI, hosting and storage resources, triaged into your remediation process instead of a separate report nobody reads.
- Traffic off the public internet. Private endpoints and network restrictions on Microsoft Foundry, Azure AI Search and the storage accounts behind them.
- 04
Adoption and enablement
Microsoft tooling puts AI inside applications people already have open, which helps adoption and hides poor value equally well. We plan for both.
- Enablement where the work happens. Microsoft 365 Copilot inside Word, Outlook and Excel, and the Copilot Studio agent published into the Teams channel that team already lives in.
- Licences allocated by task fit. Start with the roles whose daily work matches measured strengths, with a review point agreed before the next tranche is assigned.
- Scenarios in your own words. The five to 10 tasks each role performs most often, written in your terminology with the phrasing that actually works.
- A feedback path someone reads. Reporting from inside the agent into a Teams or Power Apps form, triaged weekly through the pilot and monthly afterwards.
- Adoption measured against the baseline. Microsoft 365 admin usage data compared with the baseline agreed at assessment, reported to the sponsor rather than to us.
- Published rules for wrong answers. What staff do when an answer is wrong, how they report it, and what must never be pasted into a prompt.
- 05
Managed continuation
Azure AI services change quickly, and so does the content underneath a grounded assistant. Continuation keeps both under observation and keeps you informed of what changed.
- Scheduled evaluation runs. The agreed question set re-run on a cadence, trended over time, with any regression raised to a named owner on your side.
- Thresholds set at design time. Azure Monitor and Application Insights covering latency, failure rate, consumption per feature and retrieval quality, alerting on limits agreed up front.
- Index health and permission drift. Failed ingestion, stale documents and access changes following staff movements or project closures, checked rather than assumed.
- Deprecation handled before the date. Foundry model version tracking with a tested migration completed ahead of a retirement date, not after an outage.
- Cost split so it can be judged. Foundry consumption, Azure AI Search tier, application hosting and Copilot licences reported separately against the original business case.
- One recommendation a month. A sponsor report covering usage, answer quality, incidents and spend, ending in the single improvement we suggest next.
Reference architecture
A grounded assistant on Azure, end to end.
How the pieces fit together on Microsoft Azure. Every engagement adapts this, and we will tell you which layers you already have.
- 01
Sources
Where the answers already live, with their permissions attached.
- SharePoint and OneDrive
- Azure SQL Database
- Line-of-business APIs
- 02
Ground
Chunked, embedded and permission-filtered before a model sees anything.
- Azure AI Search
- Azure Functions
- Purview sensitivity labels
- 03
Reason
Model choice, content filters and tool boundaries set per application.
- Microsoft Foundry
- Foundry Models
- Foundry Agent Service
- 04
Serve
Every call authenticated, throttled and logged before it reaches a person.
- Azure API Management
- Microsoft Copilot Studio
- Azure App Service
Across every layer
- Microsoft Entra ID for identity and security trimming
- Azure Policy for guardrails
- Azure Monitor and Application Insights for evaluation and cost
- A named human approval before any write to a business system
Technology reference
The Microsoft Azure services we build with.
A reference architecture view of the platform services used in this domain, and what each one does in the design.
Model and agent platform
Microsoft FoundryThe build and deployment surface for custom AI applications, covering model deployment, content filters, evaluation runs and per-application quota in one place.
Foundry ModelsThe model catalogue we benchmark against your own tasks, so selection is made on measured accuracy, cost and latency rather than on reputation.
Microsoft Copilot StudioBusiness-facing agents and topic flows published into Teams or a website, with connectors into line-of-business systems and approval steps before consequential actions.
Microsoft 365 CopilotTask assistance inside Word, Outlook, Excel and Teams, allocated to the roles whose daily work genuinely matches what it does well.
Grounding, data and machine learning
Azure AI SearchThe retrieval layer behind grounded answers, with hybrid and semantic ranking plus security trimming so results respect each user's existing permissions.
Microsoft FabricThe governed source of analytical data used for grounding and for any metric an assistant is asked about, keeping AI answers consistent with your reporting.
Azure Machine LearningForecasting and classification models with versioned datasets, a registered model store and a documented promotion path into production.
Identity, governance and protection
Microsoft Entra IDOne access model for users, applications and managed identities, so an assistant can only return what the signed-in person is already entitled to see.
Microsoft PurviewSensitivity labelling, classification and lineage across the content in scope, used to decide what may be indexed and what must stay out.
Microsoft Defender for CloudPosture monitoring and recommendations across AI, hosting and storage resources, with findings routed into the remediation process you already run.
Runtime and integration
Azure FunctionsEvent-driven processing for document extraction, enrichment and index updates, scaled by workload instead of provisioned in advance.
Azure App ServiceHosting for assistant front ends and APIs where a managed platform is the right fit and container orchestration would only add overhead.
Azure Kubernetes ServiceHosting where you already operate a cluster or need fine control over networking, scaling and workload isolation for the AI application.
Azure API ManagementA single governed entry point to model and tool endpoints, providing authentication, throttling, per-consumer quota and consistent request logging.
Observability and business automation
Azure MonitorPlatform metrics, alert rules and log retention across the AI workload and every resource it depends on.
Application InsightsRequest-level tracing through prompt, retrieval and tool calls, which is what makes a poor answer diagnosable instead of anecdotal.
Power AutomateApproval routing and system-to-system steps around an AI decision, so an extraction becomes an action in the correct business system.
Power AppsLightweight review and exception-handling screens where a person must confirm, correct or reject what the model produced.
Product names and icons are trademarks of Microsoft and Amazon Web Services, reproduced unmodified from their official architecture icon libraries to identify the technologies used in these architectures. Their presence does not indicate partnership, certification or endorsement by either vendor.
Azure questions
What people ask about doing this on Azure.
Cost, lock-in and the parts that go wrong, answered before you have to ask twice.
Do we need Microsoft 365 Copilot licences as well as Azure consumption?
Not necessarily, and this is usually the first place money gets wasted. Microsoft 365 Copilot is per seat and works across content people already have in Microsoft 365. A Foundry application is per call and suits a defined workflow with a lot of volume. We cost both against each use case during assessment, and for some clients the answer is a smaller number of Copilot licences plus one custom application, not both at full scale.
Will our data be used to train Microsoft's models?
Under the current Azure OpenAI and Microsoft Foundry terms, your prompts and outputs are not used to train the foundation models. We confirm the specific terms attached to the services in your architecture at design time and record them, because vendor data-handling terms change and a term you verified 18 months ago is not evidence today. Where a client needs that in writing for an audit committee, the record is what they take to it.
What does this actually cost to run each month?
It depends on volume, and any consultancy that answers with a single figure before seeing your workload is guessing. The cost model has four separable parts: Foundry consumption per call, the Azure AI Search tier, application hosting, and any Copilot licences. We estimate each at expected and peak volume during assessment and state every assumption, so when the first invoice arrives you can see which assumption was wrong rather than just that the total was.
We already pay for Copilot and nobody uses it. Why would this be different?
Usually because the problem was never the model. Low Copilot value is most often permission sprawl, duplicated and stale content, and licences assigned to roles whose daily work the tool does not touch. We look at usage by team and the tasks people were actually attempting before recommending anything new. Often the fix is content clean-up and reallocating licences, and we will say so even though it is a smaller piece of work for us.
Could it show someone a document they are not allowed to see?
That is the failure that ends projects, so it is tested rather than asserted. Retrieval is trimmed by Entra ID group so the index cannot return what the person asking could not already open, and we prove it by running the same queries as accounts at three permission levels, including one that should see nothing. If your existing SharePoint permissions are themselves wrong, an assistant will surface that fast, which is why the tenant review comes first.
What happens when Microsoft renames or retires the model we built on?
It will, and more than once. The architecture is kept deliberately model-swappable, prompts and tool definitions live in source control, and your evaluation set is what proves a replacement model is at least as good before it goes live. Under continuation we track version and deprecation notices and complete a tested migration before the retirement date rather than after something breaks.
Related industries
Where this work has the most leverage.
Professional Services
Governed enterprise search, document intelligence and secure copilots that respect matter confidentiality and conflict boundaries.
Construction and Property
Tender intelligence, addenda tracking and project reporting that keep estimators and contract administrators ahead of the documents instead of buried in them.
Healthcare and Community Services
Administrative automation, policy search, workforce analytics and privacy uplift for healthcare and community providers, with clinical decisions left entirely to clinicians.
Free discovery workshop
Start with a artificial intelligence discovery workshop.
Bring one challenge. We will assess whether Microsoft Azure is the right platform for it before recommending anything.