Security and compliance
ACLs, ABAC with governed tags, anonymisation, PII controls and purging for erasure requests.
This section tests whether you can lock data down with the smallest set of rights, protect sensitive columns and rows at scale, and then truly delete data when the law says so. The biggest way people lose marks is stopping one step early: a DELETE statement or a mask feels like the end, but old file versions and unmasked copies often still hold the data.
Walk the guide, line by line
Each row is one line of the official guide, in its own words on the left. Read across: plain words, what a question looks like, the answer, and the trap.
| Guide line | In plain words | A question looks like | The answer | The trap |
|---|---|---|---|---|
| 6.1 Apply least-privilege access control lists (ACLs) to secure Unity Catalog securable objects and workspace resources. | Least privilege means giving each person or group only the rights they need and nothing more. In Unity Catalog, rights are called privileges, and they are granted on securable objects (catalogs, schemas, tables, volumes, functions). Workspace resources such as clusters, jobs and secrets have their own access control lists. | A group of analysts must read one table and nothing else in a shared catalog. The question asks which grants give exactly that. | Grant USE CATALOG on the catalog, USE SCHEMA on the schema and SELECT on the table, to a group rather than to individual users. USE CATALOG and USE SCHEMA only let a user reach objects and give no data access by themselves. Creation privileges such as CREATE TABLE are best granted at schema level, and EXECUTE on individual functions rather than a whole schema. Try it: Create a schema with two tables, grant a test group USE CATALOG, USE SCHEMA and SELECT on one table, and use SHOW GRANTS to confirm what is visible. | Granting SELECT on the whole catalog is quick and works, but it also exposes every current and future table, because privileges are inherited. Granting ALL PRIVILEGES is wider still. Runner-up wins when: Granting SELECT at schema level wins when the analysts really do need every table in that schema, including tables added later. |
| 6.2 Apply attribute-based access control (ABAC) policies with governed tags to enforce row filters and column masks at scale. | Attribute-based access control (ABAC) decides access from labels called governed tags attached to data, instead of writing one rule per table. A policy says: for any column carrying this tag, apply this mask or filter to these groups. Governed tags are account-level tags with controlled allowed values, and only people with the ASSIGN permission can apply them. | A bank has hundreds of tables with email columns spread across many schemas, and wants one rule that masks every email column for analysts but not for data owners. | Tag the columns with a governed tag, write one masking function, and create one policy with CREATE POLICY on the catalog or schema, using MATCH COLUMNS has_tag_value to select the columns, TO for the affected group and EXCEPT for the exempt one. You need MANAGE on the object where the policy attaches and EXECUTE on the function, and compute must be serverless or Databricks Runtime 16.4 or above. Tags on a catalog, schema or table are inherited by objects below for policy evaluation, but not down to columns. Try it: Check the docs for current availability in your workspace first; if available, tag one column and create a schema-level column mask policy. | Setting a column mask with ALTER TABLE on each table works, but it must be repeated for every table and every new table, so it does not scale and drifts. Runner-up wins when: Per-table SET MASK wins when only one or two tables need protecting and no governed tags exist yet. |
| 6.3 Apply anonymization and pseudonymization methods — such as hashing, tokenization, suppression, and generalization — to confidential data (e.g., using column masks and related Unity Catalog features). | Anonymization removes the link to a person for good; pseudonymization swaps identifiers for stand-ins that can be linked back with extra information. Hashing turns a value into a fixed code, tokenization swaps it for a random token kept in a lookup, suppression blanks it out, and generalization makes it less precise, such as an age band instead of a birth date. | Analysts need to count customers per postcode region and join two tables on customer id, but must not see names or exact birth dates. | Use a column mask that hashes the customer id (for example with sha2 and a secret salt) so the join key stays consistent, suppress the name, and generalize birth date to a year or age band. A column mask is a SQL function attached to a column that returns a replacement value, and each column can have one mask. Hashing without a salt on a small value space can be reversed by guessing, so salt it. | Suppressing the customer id with NULL looks safest but breaks the join, so the analysts cannot do the stated work. Runner-up wins when: Full suppression wins when the field is never needed for analysis, such as a free-text name. |
| 6.4 Implement a compliant data pipeline that enforces PII detection and masking controls across batch and streaming workloads using Unity Catalog features. | A compliant pipeline finds personal data (PII, personally identifiable information) as it flows in and makes sure it is masked or restricted wherever it lands, for both batch loads and streaming. The controls live in Unity Catalog, so they follow the table, not the job. | A pipeline loads customer files in batch and from a stream into the same bronze table. Compliance wants new PII columns found automatically and masked for most users without changing each job. | Land both paths into Unity Catalog tables, let data classification tag detected sensitive columns with system governed tags, and attach an ABAC mask policy keyed on those tags. Because the policy sits on the governed table, batch and streaming writes are covered by the same control. Data classification scans new tables and columns incrementally, typically within 24 hours, so it is not instant. | Masking inside each pipeline's code covers only that job, and a new job or a manual write bypasses it. Runner-up wins when: In-pipeline masking wins when the raw value must never be stored at all, for example when the landing zone itself is out of compliance scope. |
| 6.5 Implement a data purging strategy that satisfies data retention and deletion policies (e.g., GDPR right-to-erasure, GDPR right-to-be-forgotten deletion) using Delta Lake and Unity Catalog features. | A purging strategy deletes personal data when a person asks for erasure or a retention period ends. In Delta Lake a DELETE only stops the rows appearing in the current version; the old files remain readable through table history until they are cleaned up. | A customer exercises their right to erasure. Rows were deleted from a table that uses deletion vectors, but legal asks for proof the data is physically gone from storage. | After DELETE, run REORG TABLE with APPLY (PURGE) to rewrite files that hold soft-deleted rows, then run VACUUM so older file versions past the retention threshold are removed. Delta keeps deleted records in history for 30 days by default, and VACUUM by default shortens time travel to 7 days. For many requests at once, MERGE against a table of deletion requests, and propagate deletes from bronze to silver and gold. Try it: Enable delta.enableDeletionVectors on a small table, DELETE a row, then run REORG TABLE APPLY (PURGE) and VACUUM, noting retention limits. | Stopping after DELETE leaves the data in earlier file versions and in time travel, so it is hidden but not erased. Runner-up wins when: Pseudonymizing instead of deleting wins only when the law and policy accept it; the docs call complete erasure the safest option. |
Mind map: Choosing the access control mechanism
People assume all access control is one thing, but who may touch an object, which columns and rows they see, and what they see inside a column are three separate controls.
| What you observe | What it points to | What to do (and when the runner-up wins) |
|---|---|---|
| Group must reach one table only | Privileges: USE CATALOG, USE SCHEMA, SELECT | Grant narrowly to a group; schema-level SELECT wins if every table in the schema is needed |
| Same sensitive column type across hundreds of tables | ABAC policy with governed tags | One policy on catalog or schema; per-table SET MASK wins if only a couple of tables are involved |
| Different groups must see different rows of one table | Row filter (function returning true or false) | Attach with SET ROW FILTER, or an ABAC row-filter policy when many tables share the rule |
| Users should see the shape but not the value | Column mask | Hash to keep joins working; suppress with NULL when the value is never needed |
Mind map: Erasure is a chain, not one statement
A successful DELETE does not mean the data is gone; it only means the current version no longer shows it.
| What you observe | What it points to | What to do (and when the runner-up wins) |
|---|---|---|
| Row deleted but still visible with time travel | Old versions still in history | Run VACUUM after the retention period; shorten retention only if policy allows |
| Deletion vectors on and files still hold the row | Soft delete only marks the row | REORG TABLE APPLY (PURGE), then VACUUM |
| Silver and gold tables still hold the person | Delete reached bronze only | Propagate deletes downstream; materialized views follow the source, streaming tables need handling |
| Regulator accepts pseudonymized data | Erasure not required in full | Pseudonymization can win here; otherwise erase completely |