The privacy decision begins before the prompt
An employee can expose information to an AI service without intending to share a database. A pasted support thread, a document attachment, or a connector to a shared drive may contain personal information far beyond the immediate question.
The useful question is not “Is AI safe?” It is “Is this data flow appropriate for this purpose, service, configuration, and group of users?” Answering that requires a practical inventory and evidence, not a general promise about the technology.
NIST's Generative AI Profile identifies privacy risks including exposure of sensitive information, unauthorized use, and harmful inference. Those risks depend on how a system is developed and used; the profile is a voluntary risk-management reference, not a conclusion that every AI service behaves identically.1
Map five places where information can travel
Begin with the actual workflow, including automated connections. A policy limited to “do not paste sensitive data” can miss repository access and saved output.
| Location | What to inspect |
|---|---|
| Input | Prompts, attachments, screenshots, audio, and copied records |
| Connected sources | Drives, mailboxes, customer systems, and retrieval indexes |
| Processing and storage | Provider access, logs, retention settings, and subprocessors |
| Output | Generated documents, inferred information, and copied summaries |
| Sharing and reuse | Team history, public links, exports, and later training or improvement uses |
Record who controls each stage and which facts remain unknown. “The provider does not train on our prompts” does not by itself explain logging, retention, support access, third-party connectors, or how colleagues can see the output.
Avoid assuming the opposite as well. Not every provider trains on every customer prompt. Verify the specific product, account type, contract, settings, and date of the claim.
Identify the purpose and the people affected
Write a narrow purpose statement before choosing data. “Summarize recurring product issues from a synthetic sample” requires less information than “analyze every support conversation with each customer's identity attached.”
Where applicable privacy law governs the processing, determine the relevant obligations rather than treating company approval as legal permission. The UK's ICO guidance emphasizes identifying distinct processing purposes and an appropriate lawful basis before processing personal data. It also notes additional conditions for special-category data where relevant. The guidance is under review following UK legislative changes, so verify current requirements for the particular use.2
For EU GDPR work, the EDPB's small-business guidance discusses minimization, privacy by design, processing records, and screening for a data protection impact assessment when processing is likely to create high risk. These are jurisdiction-specific considerations; they should not be applied indiscriminately as a statement of every country's law.3
The GDPR applicability checklist offers a starting point for scope questions. Employee consent, a vendor contract, or a privacy notice is not a universal shortcut around the applicable analysis.
Use an original pre-sharing checklist
For a proposed AI workflow, complete this decision record:
- Purpose: What concrete task requires AI, and what alternative was considered?
- Data: What personal, confidential, regulated, or contractual information is involved?
- Minimum necessary input: Which fields and documents can be removed or replaced?
- Authority: Who approves the purpose, and who confirms applicable legal and contractual restrictions?
- Provider and account: Which exact service and configuration will be used?
- Retention and reuse: What is saved, for how long, by whom, and for which purposes?
- Access: Who can read the inputs, history, connected data, and outputs?
- Testing: What synthetic tests will examine access, disclosure, deletion, and errors?
- Human review: Who checks the output before any decision or wider disclosure?
- Stop and reassessment: What change or incident causes the use to pause?
Add supporting records and a review date. If a required answer is unknown, leave it visibly unknown and assign an owner. Do not convert “we have not found evidence” into “the provider does not do that.”
A fictional customer-support example
A fictional software company wants to summarize recurring complaints. Its first proposal exports all support tickets, including names, email addresses, billing details, and free-text attachments, into a general chat account.
The team narrows the purpose to identifying product themes. It creates a synthetic sample to test the task, then investigates whether a minimized real dataset is appropriate. The following original record compares the initial idea with a safer proposed workflow:
| Decision | Initial idea | Revised proposal to evaluate |
|---|---|---|
| Dataset | Entire ticket archive | Selected issue descriptions with unnecessary fields removed |
| Identity | Customer names and contact details | No direct identifiers where not needed |
| Attachments | Included automatically | Excluded unless separately justified |
| Access | Shared account history | Approved users and controlled output location |
| Output | Sent directly to leadership | Human checks accuracy and sensitive content first |
| Retention | Not investigated | Confirmed setting, contractual terms, and removal process |
The revised proposal is not automatically approved. Free text may still identify a person or disclose sensitive facts. The privacy reviewer checks residual information and provider terms before deciding whether the real-data workflow may proceed.
The example illustrates why removing a name alone is insufficient. A unique incident, a rare role, a location, or a detailed narrative can make information linkable to an individual.
Check connectors and retrieval boundaries
An AI assistant connected to a repository can introduce a different exposure path from manually uploading one document. Determine whether it respects the underlying permissions, what it indexes, and how quickly permission changes take effect.
Use synthetic accounts and documents to test useful denial cases: a user outside the authorized group asks for a restricted file; a previously authorized user is removed; a shared link is revoked; an output requests information from another organization. The expected result must be defined before the test.
Do not assume a convenient answer proves authorized access. The system should preserve the relevant permission boundary, and the test record should identify what was actually observed. Keep tests small and avoid using real sensitive data to discover whether a boundary exists.
Review the output as a separate disclosure
Generated text can repeat input data or infer sensitive information. It can also invent facts about a person. A fluent summary is not a reliable privacy or accuracy check.
Before sharing output, verify the source, intended audience, unnecessary details, and whether the conclusion is supported. If the task concerns individuals or consequential decisions, obtain appropriate additional review and confirm applicable requirements.
For routine internal summaries, use a practical release rule: a named reviewer checks the output against the approved source material, removes unnecessary personal details, and records material uncertainties. This is an example of process design, not a guarantee against disclosure.
Reassess when the service or purpose changes
A provider can change terms, model behavior, retention options, subprocessors, or connector functionality. A team can also expand a harmless-looking drafting use into profiling or decision support. Both kinds of change deserve renewed review.
Define change triggers in the approval record. Examples include a new connector, additional data categories, broader sharing, a new provider account type, or a purpose involving decisions about people. Keep the old decision and the new one so the team can explain what changed.
If inappropriate sharing occurs, stop the affected flow, preserve relevant records, and follow the organization's response process. Do not improvise deletion or notification decisions without the responsible team's involvement.
Frequently asked questions
Is redacting names enough?
Not necessarily. Other details can identify or reveal information about a person. Evaluate the remaining content and whether the task can use synthetic or more limited information.
Does a paid AI account make every use acceptable?
No. Account type may affect terms and settings, but purpose, data, access, applicable obligations, and output handling still need review.
Can staff use AI to summarize internal documents?
That depends on the documents and approved workflow. Start with a defined purpose, authorized service, minimized information, and explicit access boundaries. The AI acceptable-use policy template can connect these checks to employee guidance.
Does this checklist establish legal compliance?
No. It organizes a privacy decision and its evidence. Legal applicability and the adequacy of the chosen safeguards require assessment of the organization's actual circumstances.
Sources and references
-
NIST. AI 600-1: Generative Artificial Intelligence Profile (2024-07-26). Voluntary companion profile; supports discussion of privacy, unreliable outputs and acceptable-use governance. ↩
-
UK Information Commissioner’s Office. How do we ensure lawfulness in AI? (2024-10-28). UK guidance currently under review after the Data (Use and Access) Act; used for limited lawful-purpose planning, not an EU/US legal conclusion. ↩
-
European Data Protection Board. Small-business guide: Be compliant. Practical EU guidance on design/default, processing records and DPIA triggers. ↩