Skip to main content

Data Governance and Compliance on AWS

Governance assembled from services, not bought as one product

There is no single AWS product that stands opposite Microsoft Purview, and being clear about that early is the difference between a programme that lands and one that is underscoped. The same outcomes are assembled from a technical catalogue, a business catalogue, a fine-grained access layer, a sensitive data discovery service and a configuration compliance service. Each has its own permission model and console. What you gain is genuinely better fine-grained access control over lake data. What you do not get is protection that travels with a document.

Why AWS

When this is the right platform.

  • Fine-grained access control on lake data is stronger and more coherent. Permissions at table, column, row and cell level plus tag-based access control are enforced consistently across the query and processing engines.
  • Account-level isolation gives you a hard boundary for separating regulated data estates from everything else, rather than relying on permissions inside one tenant.
  • AWS Config conformance packs ship templates keyed to Australian frameworks, including Essential Eight and ISM operational best practices, which is a useful starting point for detective controls.
  • Security Lake normalises logs from AWS, on-premises and third-party sources into an open schema in buckets you own, with retention you control.
  • Catalogue and access metadata are queryable with standard SQL, which makes producing an evidence extract for an auditor a practical exercise rather than a screenshot task.

Where it is less suited

We would rather say this now than after a migration.

  • There is no single product opposite Microsoft Purview. Governance is assembled from a technical catalogue, a business catalogue, fine-grained access, sensitive data discovery and configuration compliance, each with its own permission model and console, which means more integration work and more places a control can quietly lapse.
  • Compliance evidence management is the weakest area and it is narrowing rather than widening. AWS Audit Manager is in maintenance mode, cannot be set up in new accounts from 30 April 2026, and AWS directs customers to third-party products for end-to-end evidence automation.
  • Sensitive data discovery is object-storage centric. Classifying personal information held in a data warehouse, relational database or SaaS source needs separate tooling and a separate cost case.
  • No protection travels with the document. There is no equivalent of a sensitivity label that stays attached after export, and no data loss prevention across collaboration and email, so the productivity estate where much personal information sits stays outside this design.
  • The business catalogue is in transition. Documentation spans both the standalone service and the newer studio environment, existing domains need an upgrade step, and a client who invested in the earlier shape will reasonably ask how long the current one lasts.
  • Conformance packs are a detective baseline, not compliance. AWS states its templates are not designed to ensure compliance with a standard and cannot guarantee passing an assessment, and nothing in a pack verifies that a policy document exists.

Business outcomes

What AWS delivers here.

Fine-grained access actually enforced
Permissions at table, column, row and cell level plus tag-based access control, applied consistently across the engines that read the lake rather than per tool.Agreed measure: Effective access to a named sensitive table verified by querying as accounts at several permission levels.
Sensitive data located in object storage
Automated discovery using managed and custom identifiers, with results retained as records usable in a privacy investigation or audit.Agreed measure: Proportion of in-scope buckets scanned, with stores outside the discovery service's reach listed explicitly.
A catalogue the business can navigate
Domains, projects, glossaries and subscription workflows so an analyst can find and request data without a ticket to the platform team.Agreed measure: Share of published data assets with a named owner and a business definition rather than only technical metadata.
Detective controls deployed organisation-wide
Configuration rules grouped into framework-keyed packs and deployed across accounts as one entity, with an organisational view of what is non-conforming.Agreed measure: Control coverage per account and region, with gaps reported rather than averaged away.
Evidence you can query
Security and access telemetry centralised in an open schema in storage you own, so producing an auditor extract is a SQL query rather than a manual exercise.Agreed measure: Turnaround to produce an agreed evidence extract, baselined against your current process.
Residency held deliberately
Log aggregation and rollup regions pinned on purpose, because these layers are where data leaves the country without anyone deciding that it should.Agreed measure: Verified location of data, metadata and log stores, re-checked whenever configuration changes.

Common client problems

What we usually hear first.

  • What is the AWS equivalent of Microsoft Purview?

    There isn't one, and any proposal that implies otherwise is underscoping the work. You assemble the technical metastore, a business catalogue, the fine-grained access layer, sensitive data discovery and configuration compliance from separate services, each with its own permission model and console. The outcomes are achievable and the integration effort is genuinely higher. We would rather quote that honestly than discover it at the halfway point.

  • We were going to use AWS Audit Manager for evidence collection.

    Not for a new build. It is in maintenance mode with no new frameworks coming, and from 30 April 2026 it cannot be set up in new accounts, so it belongs in migration conversations with existing users rather than in a new design. The current starting point is configuration conformance packs plus a third-party evidence tool. AWS itself directs customers to partner products for end-to-end evidence automation, which tells you where the gap is.

  • Are we compliant if we deploy the Essential Eight conformance pack?

    No, and AWS says so itself: the templates are not designed to ensure compliance with a governance standard and cannot guarantee passing an assessment. A pack covers only the controls a rule can evaluate. Nothing in it checks that a policy document exists, that staff were trained, or that someone reviews the output. It is a useful detective baseline and it is not a compliance position.

  • Our Lake Formation rollout is producing random-looking denials.

    They are not random, they are the dual permission model. Every request must satisfy both the identity policy and the Lake Formation grant, so a half-migrated set of catalogue permissions produces confusing denials and, more dangerously, paths that remain over-permitted. We map the effective permission for each principal across both layers, complete the migration rather than leaving it partial, and verify by querying as real accounts.

  • Can we classify personal information in Redshift and RDS, not just S3?

    Not with the same service. Sensitive data discovery on AWS is heavily object-storage centric, so classifying personal information held in a data warehouse, relational database or SaaS source needs separate tooling and a separate cost case. This is a real gap rather than a configuration detail, and we scope it explicitly rather than letting a client assume one service covers the estate.

  • We invested in a business catalogue. How long will the current shape last?

    A fair question and the honest answer involves uncertainty. The business catalogue capability is now used within a broader studio environment, existing domains have an upgrade step into it, and documentation spans both shapes. We found no deprecation notice, but the direction of travel reads like a transition. We design so your glossary, metadata forms and ownership records are the durable asset rather than the console you happen to view them in.

How we deliver

Our AWS delivery approach.

  1. 01

    Assessment and advisory

    The AWS assessment spends most of its time on what is genuinely coverable, because the gaps here are service boundaries rather than configuration choices.

    • Coverage gaps named up front. Which stores the discovery service can reach and which need separate tooling, since object storage focus is a real limit rather than a setting.
    • Dual permission model mapped. Effective access per principal across both the identity policy and the catalogue grant, because a partial migration produces both denials and over-permission.
    • Evidence approach decided. Conformance packs plus a third-party evidence tool scoped realistically, given the platform's own evidence automation is in maintenance mode.
    • Framework packs matched to obligations. Which shipped templates correspond to what genuinely binds you, and which of your obligations no rule can evaluate at all.
    • Catalogue transition risk assessed. Where the business catalogue capability is heading, so the durable asset is your glossary and ownership records rather than a console.
    • Rollup region residency. Log aggregation and rollup regions checked, since this is the layer that moves data outside Australian regions without a deliberate decision.
  2. 02

    Architecture and implementation

    Catalogue and access first, because the dual permission model is the thing that goes wrong. Discovery and evidence follow, with the gaps in coverage stated rather than glossed.

    • Technical metastore as the foundation. Schema and partition metadata held centrally so the access and query layers resolve against one source rather than several.
    • Fine-grained grants, fully migrated. Table, column, row and cell permissions plus tag-based access control completed rather than left half-applied alongside legacy grants.
    • Business catalogue with subscriptions. Domains, projects, glossary terms and request workflows so analysts find and request data without a platform ticket.
    • Discovery scoped and costed. Managed and custom identifiers applied across object storage, with results retained as records usable in an investigation.
    • Conformance packs at organisation scope. Framework-keyed detective controls deployed across accounts as one entity rather than configured account by account.
    • Telemetry normalised into your storage. Security and access logs centralised in an open schema in buckets you own, queryable with SQL for an evidence extract.
  3. 03

    Security and governance

    The AWS strength here is precision of access control. The weakness is that nothing protects content once it leaves the platform, so the design has to acknowledge where governance simply stops.

    • Both permission layers designed together. Identity policy and catalogue grants treated as one design problem, because a request must satisfy both and a gap in either is invisible.
    • Row and column controls verified. Enforcement tested by querying as accounts at several permission levels rather than inferred from the grants that were applied.
    • Workforce identity, not shared roles. Access traced to a person through permission sets rather than a shared role, so an audit entry names someone.
    • Customer-managed keys throughout. Keys for governed stores and evidence repositories under your control, with key access separated from the teams being audited.
    • The export boundary acknowledged. No protection travels with a downloaded file, so controls concentrate on preventing extraction rather than on protecting content afterwards.
    • Retention set per source. Log retention chosen by source and scenario, since centralised telemetry grows with every account and region added.
  4. 04

    Adoption and enablement

    Because governance is assembled here, the main adoption risk is a control quietly lapsing in one of the several consoles nobody has opened for a quarter.

    • Ownership across several consoles. A named owner for each service in the governance stack, because assembled governance fails where responsibility is ambiguous.
    • Analysts taught the request path. How to find data in the catalogue and subscribe to it, so the governed route is faster than asking someone for credentials.
    • Platform team trained on both layers. The dual permission model taught explicitly, since this is the single largest source of support tickets after go-live.
    • Coverage limits communicated. Which stores are outside discovery scope written down and shared, so nobody assumes the estate is fully covered.
    • Evidence extract rehearsed. The auditor extract produced once before it is needed, which is when a missing log source becomes obvious.
    • Runbooks per control. What to do when a conformance rule reports non-compliance, including who decides whether it is a genuine failure.
  5. 05

    Managed continuation

    Assembled governance decays at the joins. Continuation watches the seams between services, new accounts and regions, and the parts of the platform that are being repositioned.

    • Control coverage per account and region. New accounts and newly opted-in regions checked against the intended pack, since coverage varies by region and a gap is real.
    • Permission drift across both layers. Identity policies and catalogue grants reviewed together, because drift in either produces over-permission that no single console shows.
    • Discovery re-run as storage grows. New buckets and prefixes scanned as they appear, with anything outside service scope still tracked as a known gap.
    • Service repositioning tracked. Catalogue and evidence services watched for transition, since one is already in maintenance mode and another is being folded into a studio.
    • Log and retention cost reported. Centralised telemetry growth reported monthly, because it scales with account count rather than with usage.
    • Residency re-verified after change. Rollup and aggregation regions re-checked whenever configuration changes, as this drift is the least visible and most asked about.

Reference architecture

Data governance on AWS, layer by layer.

How the pieces fit together on AWS. Every engagement adapts this, and we will tell you which layers you already have.

  1. 01

    Data estate

    Where governed data lives, across lake, warehouse and operational stores.

    • Amazon S3
    • Amazon Redshift
    • Amazon DynamoDB
  2. 02

    Discover and catalogue

    Technical metastore, business catalogue and sensitive data discovery.

    • AWS Glue Data Catalog
    • Amazon DataZone
    • Amazon Macie
  3. 03

    Enforce access

    Both the identity policy and the catalogue grant must permit a request.

    • AWS Lake Formation
    • IAM Identity Center
    • Tag-based access control
  4. 04

    Evidence

    Normalised logs and detective controls, queryable for an audit extract.

    • Amazon Security Lake
    • AWS CloudTrail
    • AWS Config conformance packs

Across every layer

  • Customer-managed keys across governed and evidence stores
  • Detective controls deployed at organisation scope
  • Rollup and aggregation regions pinned for residency
  • No protection travels with an exported file
Count the products in layers 02 and 03 and compare them with the single product on the Azure equivalent. That difference is the integration effort, and it is the line most often missing from an AWS governance proposal.

Technology reference

The AWS services we build with.

A reference architecture view of the platform services used in this domain, and what each one does in the design.

Catalogue and discovery

  • AWS GlueThe Glue Data Catalog is the technical metastore holding schema and partition metadata that the access and query layers resolve against.
  • Amazon DataZoneBusiness catalogue with domains, projects, glossary terms, metadata forms and subscription workflows, and the foundation the newer studio catalogue is built on.
  • Amazon MacieSensitive data discovery across object storage using managed and custom identifiers, retaining results as records usable in an investigation.

Access control

  • AWS Lake FormationFine-grained permissions at table, column, row and cell level plus tag-based access control, enforced across the query and processing engines.
  • AWS Identity and Access ManagementThe second half of the dual permission model. A request must satisfy both the identity policy and the catalogue grant, so both are designed together.
  • IAM Identity CenterWorkforce identity and permission sets, so data access traces to a person rather than a shared role.
  • AWS KMSCustomer-managed keys for governed stores and evidence repositories, with key access separated from the teams being audited.

Controls and organisation scope

  • AWS ConfigConfiguration recording and rule evaluation. Conformance packs deploy framework-keyed detective controls across an organisation, including Australian operational best-practice templates.
  • AWS OrganizationsThe boundary for organisation conformance packs and delegated administration, so controls are set once rather than per account.
  • AWS Control TowerFramework-keyed preventive, proactive and detective controls deployable to organisational units, with a view of non-conforming resources.

Evidence and audit

  • AWS CloudTrailThe record of API activity across accounts, which is what lets you state who touched what.
  • Amazon Security LakeCentralises security logs from AWS, on-premises and third-party sources into buckets you own, normalised to an open schema with retention you control.
  • Amazon AthenaSQL over centralised log and configuration data, which is the practical way to produce an evidence extract for an auditor.
  • AWS ArtifactOn-demand AWS compliance documentation and third-party audit reports. Evidences AWS's side of the shared responsibility model only, never yours.

Product names and icons are trademarks of Microsoft and Amazon Web Services, reproduced unmodified from their official architecture icon libraries to identify the technologies used in these architectures. Their presence does not indicate partnership, certification or endorsement by either vendor.

AWS questions

What people ask about doing this on AWS.

Cost, lock-in and the parts that go wrong, answered before you have to ask twice.

  • What is the AWS equivalent of Microsoft Purview?

    There is no single equivalent, and that is the most important thing to know before scoping an AWS governance programme. You assemble the outcome from a technical metastore, a business catalogue, a fine-grained access layer, a sensitive data discovery service and a configuration compliance service. The access control you get is genuinely better than the Azure equivalent for lake data. The integration effort is higher and there is no protection that follows a file after export.

  • Why are we getting access denied errors after enabling Lake Formation?

    Because a request has to satisfy two independent permission systems: the identity policy and the Lake Formation grant. Denials that look random are usually a partially migrated permission set, where some paths are governed by the new grants and others still rely on legacy catalogue permissions. The more serious version of the same problem is a path that stays over-permitted, which produces no error at all and is only found by testing.

  • Can AWS Config conformance packs make us Essential Eight compliant?

    No. AWS states plainly that its templates are not designed to ensure compliance with a governance standard and cannot guarantee passing an assessment. A pack evaluates the controls a rule can check, which excludes anything involving a document, a process or a person. Essential Eight is also assessed against maturity levels rather than as a pass, and much of it concerns endpoints and administrative privilege outside the cloud estate.

  • What should we use for compliance evidence now that Audit Manager is winding down?

    For a new build, configuration conformance packs for the detective controls plus a third-party evidence platform for collection and mapping. Audit Manager is in maintenance mode with no new frameworks and cannot be set up in new accounts from 30 April 2026, so designing around it now creates rework. AWS itself points customers at partner products for end-to-end evidence automation, which is a fair indication of the gap.

  • Does our data stay in Australia?

    The data usually does if you choose Australian regions. The layers that catch people out are log aggregation and rollup regions, plus any cross-region metadata or scanning configuration, which can move content outside the country unless deliberately pinned. This is normally the first question a client's own auditor asks, so we verify actual locations rather than assume them, and re-verify whenever a service or aggregation setting changes.

Related industries

Where this work has the most leverage.

  • Logistics and Warehousing

    Shipment and document automation, proof-of-delivery processing and exception dashboards that let a small team run a large network.

  • Manufacturing and Distribution

    Demand and inventory intelligence, production analytics and supplier automation built on data your planners already trust.

  • Healthcare and Community Services

    Administrative automation, policy search, workforce analytics and privacy uplift for healthcare and community providers, with clinical decisions left entirely to clinicians.

Free discovery workshop

Start with a data governance discovery workshop.

Bring one challenge. We will assess whether AWS is the right platform for it before recommending anything.