AI Doesn’t Just Read Your Personal Data. It Creates New Data About You.
AI can create new personal information by inferring things about customers, employees and applicants. Here is what business leaders need to understand.
When companies talk about personal data, they usually think about information they collected directly: a name, email address, phone number, purchase history, customer record, service conversation or date of birth.
AI makes that definition too narrow.
An artificial intelligence system does not only use personal information. It can also create it. It summarizes, classifies, scores, connects, infers and predicts. What it concludes about a person can become as sensitive as the information that person originally provided.
For business leaders, the question is therefore no longer only: “Did we send personal information to an AI system?”
It is also: “What new information is our AI creating about people?”
An inference can become personal information
Quebec’s Commission d’accès à l’information explains that personal information is information that can identify an individual directly or indirectly. It also states that information generated using artificial intelligence systems, or inferred information, can be personal information subject to applicable privacy laws.
Source: Commission d’accès à l’information du Québec — What is personal information?
That changes where privacy risk begins.
If a system concludes that a customer is likely to leave, an employee may be under distress, an applicant may be a poor fit, or a consumer may be financially vulnerable, the organization is no longer simply analyzing existing information.
It may be creating a new piece of personal information about an identifiable individual.
Canada’s privacy regulators take the same direction. Their principles for generative AI recommend treating inferences about identifiable individuals as personal information, limiting collection and use to what is necessary, and avoiding secondary uses that were not originally specified.
AI does not just read the record
In practice, AI creates this type of information more often than most organizations realize.
A tool can summarize a customer’s emails and conclude that the person is “frustrated and likely to cancel.” An agent can analyze missed appointments, late payments and support conversations, then produce a churn-risk score. An HR system can read résumés, interview notes and internal comments, then generate an assessment of leadership potential or likelihood of leaving.
The organization may believe it is simply automating the reading of existing data.
But it may actually be creating a new information object that can be stored, shared, reused or combined with other data.
AI does not only read the record. It can write a new layer of the record.
That is where governance has to follow.
A very ordinary example: customer service
Consider a small or mid-sized business using AI to support its customer-service team.
The initial objective is simple: summarize conversations, identify urgent requests and recommend the best next action.
Soon, the system begins classifying customers according to signals such as frustration, complaint risk, escalation probability, account value or churn risk.
This can be genuinely useful.
But if the company stores those assessments in its CRM, shares them across teams or uses them to prioritize customers, the outputs are no longer just temporary productivity notes. They can become personal information with a real effect on the customer relationship.
The risk increases when the inference is wrong.
A person may be labelled “difficult,” “high risk” or “low value” because a message was misunderstood, context was missing, or the model placed too much weight on a weak signal.
AI does not need to make the final decision to influence the outcome. It only needs to change how people treat the individual.
The problem: inferences look objective
A machine-generated score or label can easily appear more objective than it really is.
An inference is still a conclusion built from signals, a model, instructions and context. It can be useful without being certain.
This matters even more when AI is used in sensitive areas such as employment, credit, insurance, health, fraud detection, vulnerability assessment or financial services.
Quebec’s privacy regulator notes that information may be sensitive because of its nature or because of the context in which it is used. Medical, biometric and otherwise intimate information are examples.
An inference can therefore become more sensitive than the data that produced it.
A combination of ordinary purchasing behaviours, for example, might allow a system to infer something about a person’s health, finances or private circumstances.
Creating less data can be a security measure
Organizations are used to protecting the data they already hold.
With AI, they also have to decide what data they are willing to create.
Before storing an automatically generated note, score or profile, the business should ask whether it is truly necessary.
Canadian privacy regulators recommend limiting the collection, use and disclosure of personal information to what is needed for the identified purpose, using anonymized or de-identified data where possible, and establishing appropriate retention schedules.
That logic is especially valuable for AI.
If an inference is useful for only a few seconds while an employee prepares a response, does it really need to be stored in the CRM for three years?
If a summary can be generated without a name, address or account number, why send those details to the model?
Sometimes the best data to protect is the data you decide not to create or keep.
Five questions leaders should ask
An organization deploying AI should be able to answer five simple questions.
What inferences are our systems producing about identifiable people? Map not only the information entering the system, but also what comes out.
What are those inferences used for? Information created for one task should not automatically become permanent data reused in unrelated contexts.
Who can see it? A temporary output visible to one employee carries a different risk from a score visible across the company.
How long do we keep it? The fact that data can be stored indefinitely does not mean it should be.
How do we correct a wrong inference? The more a conclusion affects an individual, the more important it is to verify its accuracy and correct its consequences.
The next frontier of privacy
For years, privacy protection was framed mainly as a collection problem: what data do we ask for, do we have consent, and where do we store it?
AI adds another dimension.
A company can manage the initial collection correctly and later create, through analysis, information that is far more revealing than the original data.
Leaders therefore need to expand their definition of the personal-information lifecycle.
It no longer begins only when data enters the organization.
It can also begin when AI infers something new about a person.
The question is no longer simply: “What data do we hold?”
It becomes:
“What do we now know about this person only because our AI inferred it — and did we really need to know it?”