Artificial intelligence is becoming increasingly common in healthcare. AI scribes, for example, can help healthcare providers reduce administrative work by transcribing patient encounters and generating clinical notes and summaries.
But when an AI tool is handling personal health information, convenience cannot come at the expense of privacy, security or patient trust.
On January 28, 2026, the Information and Privacy Commissioner of Ontario (IPC) released AI Scribes: Checklist of Key Considerations for the Health Sector, alongside its broader guidance document, AI Scribes: Key Considerations for the Health Sector. Together, these documents provide practical direction for health information custodians considering whether to develop, procure or use AI scribes.
The checklist is not a legally binding instrument. Rather, it is intended to help custodians translate the IPC's guidance into practical questions and processes, and should be read together with the main guidance document.
For organizations operating under Ontario's Personal Health Information Protection Act, 2004 (PHIPA), the message is clear: adopting an AI scribe is not simply an IT decision. It is also a privacy, governance, security and accountability decision.
What is an AI scribe?
An AI scribe is an artificial intelligence system that can listen to or process information from a healthcare encounter and generate a transcript, medical note or summary that may then be incorporated into an electronic medical record or other health record.
The technology can offer significant administrative benefits. However, because these systems may process personal health information, their use creates privacy and security considerations that healthcare organizations need to address before implementation. The IPC's guidance takes a privacy-first approach and emphasizes that organizations should consider risks throughout the AI system's lifecycle, from development and procurement through implementation, ongoing monitoring and eventual decommissioning.
What does the IPC checklist cover?
The checklist is organized into four main parts. The first focuses on preparing an AI Governance and Accountability Framework. The second addresses organizations developing AI scribes, while the third focuses on custodians procuring AI scribes. The fourth addresses custodians using AI scribes.
All custodians considering AI should address the minimum requirements set out in Part I. The remaining sections depend on the organization's role in relation to the AI system — whether it is developing, procuring or using it. This structure is important because privacy responsibilities do not begin when an AI scribe is switched on. They begin much earlier, with the decision to introduce the technology in the first place.
1. Start with governance
Before introducing an AI scribe, custodians should have an AI governance and accountability framework in place, addressing the AI system throughout its lifecycle and integrated into the organization's existing governance structures, practices and culture.
The IPC checklist highlights the importance of having an AI governance committee, or an equivalent person or structure, with appropriate interdisciplinary representation, oversight and accountability. It also points to the need for an AI risk management framework, data minimization and purpose limitation, and Privacy Impact Assessments (PIAs) where appropriate.
Custodians should also establish comprehensive written policies, practices and procedures governing the use of AI — addressing authorized uses, responsibilities under PHIPA, agent obligations, breach notification requirements, enforcement and consequences for non-compliance. Organizations may also need AI-specific policies addressing ethics, transparency, explainability, bias, human oversight and incident response.
Governance should also extend to security safeguards, including monitoring, logging, auditing and periodic reviews, along with processes for identifying and responding to privacy breaches, including timely reporting requirements for agents and vendors.
Training and awareness are equally important. Individuals should understand the organization's AI policies, appropriate uses of AI, potential bias and discrimination risks, the importance of human oversight and the mechanisms available for reporting concerns. Confidentiality and end-user agreements should also be in place before agents are given access to personal health information or new or substantially changed AI systems.
The goal is not simply to have policies on paper. Organizations should be able to demonstrate how these policies operate in practice and how risks are identified, addressed and monitored over time. Where an organization decides not to implement a particular component of the framework, the checklist encourages custodians to document the decision and the rationale behind it.
2. Privacy Impact Assessments should be part of the process
A Privacy Impact Assessment is particularly important when introducing an AI system that involves the collection, use or disclosure of personal health information. For an AI scribe, this means looking closely at what information the system will collect, why it needs that information, where the information will be stored and who will have access to it.
The organization should also consider whether information will be transferred to a third party, what the vendor will do with the information, how long it will be retained and what happens to the information when the service ends. These questions should not be treated as a one-time exercise — findings should be documented and incorporated into the organization's broader privacy and risk management processes.
3. Choosing an AI vendor requires more than a product demonstration
For custodians procuring an AI scribe, the IPC checklist encourages a thorough assessment of the third-party vendor. A vendor's marketing claims are not a substitute for due diligence.
Organizations should understand the AI system's intended purpose and permitted uses, as well as its capabilities and limitations — including the accuracy of the system, how its model has been trained and validated, the nature and lawfulness of its training data, and the measures in place to address bias and discrimination. The assessment should also consider the vendor's security safeguards, ongoing monitoring and testing processes, and ability to respond when the system produces inaccurate, unexpected or potentially harmful outputs — including whether the vendor can pause or cease operation of the system where its performance falls below an acceptable threshold.
4. Contracts are an important privacy safeguard
The relationship between a healthcare organization and an AI vendor should be supported by appropriate contractual safeguards. Contracts should clearly establish responsibilities relating to personal health information, security, confidentiality, privacy compliance, incident and breach reporting, data handling and other relevant obligations, and provide clarity around the vendor's responsibilities when unexpected outputs, security incidents or privacy breaches occur.
Where personal health information is disclosed to or handled by a third party, custodians must consider whether the disclosure is permitted under PHIPA and whether consent or other legal requirements apply. A contract should not be treated as a formality — it is one of the mechanisms through which an organization establishes accountability and manages the risks of relying on an external AI system.
5. Human oversight remains essential
AI-generated notes are not automatically accurate simply because they were produced by an AI system. The IPC's guidance emphasizes the importance of human oversight — custodians using AI scribes should have processes in place to review AI-generated outputs for accuracy before they are relied upon, used or disclosed.
This is particularly important in healthcare, where an inaccurate transcription or summary can have consequences beyond a simple administrative error. Organizations should establish clear expectations around who is responsible for reviewing AI-generated records, what should be reviewed and what steps should be taken when an error is identified. AI may assist with documentation, but it should not replace appropriate human judgment and oversight.
6. Patients should understand when AI is being used
Patients should receive clear information about the use of an AI scribe, including its purpose, relevant risks and available alternatives. The IPC checklist also emphasizes the importance of obtaining valid, PHIPA-compliant consent where required.
Importantly, patients should not be made to feel that refusing the use of an AI scribe means they will receive a lower standard of care. Healthcare organizations should also ensure that knowledgeable individuals are available to answer questions about how the AI system operates and how personal health information is collected, used, stored and protected.
7. Do not put personal health information into unauthorized AI tools
One of the most important practical warnings from the IPC is that healthcare professionals should not enter personal health information into AI tools that have not been authorized by the health information custodian.
The IPC expressly states that entering personal health information into an unauthorized AI scribe tool can constitute a privacy breach and may trigger obligations to notify affected individuals and report the incident to the IPC. An AI tool being accessible does not mean it is approved for handling personal health information. Organizations should establish clear rules about which AI tools may be used, the purposes for which they may be used and the circumstances in which personal health information may be entered into them.
8. Monitoring cannot stop after implementation
AI governance is not a one-time exercise. AI systems can evolve — their capabilities, training, integrations and uses may change over time — so organizations should continue to monitor performance, privacy and security risks after implementation, including accuracy problems, unexpected outputs, bias or discriminatory impacts, and privacy and security incidents.
Ongoing monitoring should be supported by clear escalation processes. Organizations should be able to pause or deactivate an AI system where its performance creates unacceptable risks, and should have processes for reassessing the system when significant changes are made.
What should healthcare organizations do now?
Before implementation, organizations should have a documented AI governance and accountability framework and complete an appropriate PIA and any other relevant risk assessments. They should clearly define the purpose and permitted uses of the AI system, assess the vendor and its privacy and security practices, and establish appropriate contractual safeguards.
They should also confirm that the collection, use and disclosure of personal health information are legally permitted, develop clear patient communication and consent processes, and establish appropriate human review of AI-generated records. Security, monitoring and audit measures should be implemented, employees and agents should receive training on responsible AI use, and organizations should establish processes for reporting errors, incidents and privacy breaches.
Responsible AI starts before deployment
The growing use of AI in healthcare presents real opportunities to improve efficiency and reduce administrative burdens. But healthcare organizations also have a responsibility to protect the personal health information entrusted to them.
For healthcare organizations in Ontario, the question should not simply be, “Can we use an AI scribe?” It should also be, “Can we use it responsibly, securely and in a way that protects the people whose health information we hold?” That is where proper legal and privacy planning becomes essential.
How Ewan Legal Solutions can help
Introducing AI into a healthcare environment can raise important privacy, contractual and compliance considerations. At Ewan Legal Solutions, we provide practical legal guidance to help organizations understand their obligations and approach AI adoption with greater confidence.
This article is provided for general informational purposes based on publicly available materials from the Information and Privacy Commissioner of Ontario. It does not constitute legal advice. The application of PHIPA and other legal requirements will depend on the specific organization, AI system and circumstances. Organizations should obtain appropriate legal advice where necessary.