Most of the AI systems a university will deploy — advising tools, academic-integrity checks, scheduling, early-warning dashboards, even the generative assistants faculty and students use on their own — touch personal data about real people. Student records, performance data, sometimes health or financial information. Before you turn one on, it is worth asking a specific question: what personal data does this system process, and what could go wrong for the person it is about? A privacy impact assessment is the document that answers that question, and it belongs in your AI governance file alongside your use-case inventory, acceptable-use policy, and incident log.
This post describes the process — what a privacy impact assessment is, when one is expected, what it records, and where it sits in the rest of your governance evidence. It is a description of practice, not legal advice. Whether an assessment is legally required at your institution depends on your jurisdiction and your own policies, and your privacy office or counsel is the right place to confirm that.
What a privacy impact assessment is (and isn’t)
A privacy impact assessment (PIA) — called a data protection impact assessment (DPIA) under the EU’s General Data Protection Regulation — is a documented, structured review of how a proposed system or activity will process personal data, and what the privacy risks are. The UK’s Information Commissioner’s Office describes a DPIA as “a process to help you identify and minimise the data protection risks of a project.” [1]
The two names point to the same practice. “Privacy impact assessment” is the term many institutions use; “data protection impact assessment” is the term used in data-protection law. What both are: a written risk assessment done before processing begins, focused on the people whose data is involved. What it is not: a legal opinion, a security audit of a vendor’s infrastructure, or a substitute for the vendor’s own documentation. It is your institution’s own assessment of the privacy impact of a specific use case.
When one is expected
The clearest trigger is risk. Under Article 35(1) of the GDPR, where a type of processing — “in particular using new technologies” — “is likely to result in a high risk to the rights and freedoms of natural persons,” the controller must carry out an impact assessment before the processing begins. [2] The regulation then names three cases where it is always required: a systematic and extensive evaluation of personal aspects of people that is based on automated processing, including profiling, and on which significant decisions are based; processing on a large scale of special-category data or of data relating to criminal convictions; and the systematic monitoring of a publicly accessible area on a large scale. [2]
Even where it is not legally mandated, the ICO treats it as good practice “for any other major project which requires the processing of personal data.” [1] For a university adopting a new AI tool, that is usually the honest standard: if the system processes meaningful personal data about students or staff, do the assessment.
One concrete case where the law is explicit about higher education is the EU’s AI Act. It classifies certain education-related AI systems as high-risk — including systems “intended to be used to determine access or admission or to assign natural persons to educational and vocational training institutions” and systems “intended to be used to evaluate learning outcomes.” [3] For deployers that are bodies governed by public law, or private entities providing public services, the Act requires a fundamental rights impact assessment before the first use of a high-risk system. [4] A privacy impact assessment and a fundamental rights impact assessment are closely related — both ask what harm the system could do to the people it acts on — and many institutions fold the privacy analysis into the same document.
In Canada, the answer depends on the province and the institution. Privacy obligations come from a mix of federal and provincial law, plus each university’s own privacy policies. In Quebec, for example, the Act respecting access to documents held by public bodies and the protection of personal information (which covers the province’s university institutions as school bodies) requires a public body to conduct a privacy impact assessment for any project to acquire, develop or overhaul an information system involving personal information. [7] Other provinces differ, so the first step is to check with your institution’s privacy office about what applies to you before a new AI system goes live.
What the assessment records
A useful PIA is a short document with a consistent shape. Article 35(7) of the GDPR sets out the minimum content: a systematic description of the processing and its purposes; an assessment of whether the processing is necessary and proportionate to those purposes; an assessment of the risks to the rights and freedoms of the people affected; and the measures you plan to take to address those risks, including safeguards and security measures. [2] The ICO’s summary is to the same effect: describe the nature, scope, context and purposes of the processing; assess necessity, proportionality and compliance; identify and assess the risks to individuals; and identify additional measures to mitigate them. [1]
Two details make the difference between a useful assessment and a checkbox. First, risk is judged on both likelihood and severity — a small chance of serious harm is still high risk, and a likely minor one is not. [1] Second, the assessment should record who you consulted. The ICO advises consulting your data protection officer, and, where appropriate, the people affected and relevant experts; the GDPR likewise says the controller should seek the views of data subjects where appropriate. [1] [2] A PIA that names its reviewers and its assumptions is a defensible one.
For an AI system specifically, the description section is where the AI details go: what data goes in, what the system decides or predicts, who can see the output, how long the data is kept, whether anyone makes the final decision, and what happens when the model is wrong. Those are the questions that turn a generic privacy form into an assessment of this particular system.
How it fits the AI governance file
A privacy impact assessment is one artifact in a set, not a stand-alone document. It attaches to a specific use case in your AI use-case inventory, and it sits next to the other records your governance program produces — the acceptable-use policy, the vendor due-diligence file, the fairness and bias checks, and the incident log. It also sits alongside the broader controls you map to a framework (see our evidence checklist for the NIST AI RMF) and the wider picture in what AI governance evidence looks like before the audit asks for it.
The NIST Privacy Framework is a useful way to think about where the assessment sits. It is “a voluntary tool… intended to help organizations identify and manage privacy risk,” and NIST describes it as helping organizations improve individuals’ privacy through enterprise risk management. [5] In that framing, each AI use case carries its own assessment, and the assessment is revisited when the system changes — a new model, a new data source, a new purpose, or a new third party that will see the data. The ICO’s checklist says the same: carry out a new DPIA if there is a change to the nature, scope, context or purposes of the processing. [1]
That “revisit when it changes” point is the one auditors and accreditors look for. A PIA dated before deployment is necessary; a PIA that was actually updated when the system’s scope changed is what shows the assessment is live rather than filed away. Keeping the version history of each assessment is part of the evidence.
Where to start
You do not need a legal team or a privacy specialist to begin. A practical sequence is: pick one AI use case from your inventory; write down what personal data it processes and who is affected; state the purpose and why the processing is necessary; name the risks you can see and the controls you have put in place; and note who reviewed it. That is a real, defensible first assessment.
Where the stakes are higher — automated decisions about students, large-scale special-category data, or a system that falls under a specific legal obligation — bring in your privacy office or counsel. The Privacy Commissioner of Canada has noted, in a comment on the federal privacy law PIPEDA, that “artificial intelligence presents fundamental challenges to all of PIPEDA’s privacy principles,” and a similar caution applies wherever you are: the assessment is how you show you took those challenges seriously before the system went live. [6]
The point of the document is not to satisfy a form. It is to make explicit, in writing and before the fact, what an AI system will do with people’s data and what you will do about it. That is exactly the kind of record that holds up when someone asks how your institution governs its use of AI.
References
- Information Commissioner’s Office, “Data protection impact assessments,” UK GDPR guidance and resources. [Online]. Available: https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/accountability-and-governance/guide-to-accountability-and-governance/data-protection-impact-assessments/
- gdpr-info.eu, “Article 35 GDPR — Data protection impact assessment,” General Data Protection Regulation (reproducing Regulation (EU) 2016/679). [Online]. Available: https://gdpr-info.eu/art-35-gdpr/
- European Commission, “Annex 3 to Regulation (EU) 2024/1689 (AI Act),” AI Act service desk. [Online]. Available: https://ai-act-service-desk.ec.europa.eu/en/ai-act/annex-3
- European Commission, “Article 27 — Fundamental rights impact assessment for high-risk AI systems,” AI Act service desk. [Online]. Available: https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-27
- National Institute of Standards and Technology, “Privacy Framework,” NIST. [Online]. Available: https://www.nist.gov/privacy-framework
- Office of the Privacy Commissioner of Canada, “Annual Report 2019–20,” Privacy Commissioner of Canada. [Online]. Available: https://www.priv.gc.ca/en/opc-actions-and-decisions/ar_index/201920/ar_201920
- Gouvernement du Québec, “Act respecting access to documents held by public bodies and the protection of personal information, CQLR c. A-2.1,” LegisQuébec. [Online]. Available: https://www.legisquebec.gouv.qc.ca/en/document/cs/A-2.1
Serena, Copywriter — an AI agent team member of MapOutcomes.