By Hannah van Kolfschooten
The Council of Europe, the most important international human rights organization on the European continent, currently is drafting a Convention on Artificial Intelligence, Human Rights, Democracy and the Rule of Law (AI Convention). The Convention aims to protect fundamental rights against the harms of Artificial Intelligence (AI), and is expected to become a global leading convention, as non-European states such as the United States (U.S.) are considering becoming signatories.
As health care is one of the top industries for AI, the forthcoming AI Convention will have important implications for the protection of health and patients. This post gives a brief outline of the background, scope, and purpose of the AI Convention. It goes on to flag common human rights issues associated with medical AI and then touches upon the most important health rights implications of the current text of the AI Convention.
- Background, Scope, and Purpose
Since 1949, the Council of Europe and its human rights court have played an important role in the protection and promotion of human rights in Europe through a long list of treaties, resolutions, and court decisions (some of which relate to new technologies). The Council’s scope is much broader than the European Union’s (EU): it has 47 member states (including all 27 EU Member States) and 5 observer states (including the U.S. and Japan). The observer states are not bound by the Council of Europe but can choose to sign on to the AI Convention — committing to its rules as well.
The AI Convention, which is expected by the end of 2023, will exist in parallel to and aims to complement the EU’s forthcoming AI Act and other regulatory initiatives, including the U.S. AI Bill of Rights. The AI Act focuses on the marketing of products using AI within the EU internal market, while the AI Convention focuses on the protection of the fundamental rights of people affected by AI systems. The Convention is principle-based and will — unlike the AI Act and the AI Bill of Rights — introduce legally binding individual human rights, which also will apply to citizens of non-EU states. The exact scope of the Convention, however, is still under discussion, with states like the U.S., the U.K., and Japan advocating that it remains limited to the public sector.
- Medical AI: Challenges for Health and Human Rights
While medical AI can improve the quality and efficiency of health care, it also comes with several risks to health and human rights. System errors in widely used AI could cause mass patient injuries, for example, in AI-powered robot-assisted surgery. Unrepresentative or low-quality datasets used to train AI models may lead to incorrect medical decisions causing health damage, such as false diagnoses or wrong treatment decisions. When datasets reflect existing biases in society, specific populations face greater risk of health harms, reinforcing discrimination and health inequality. The enormous ‘data hunger’ of medical AI may also affect medical privacy, and the opaque nature of many AI applications may put existing health practices and other patients’ rights under pressure, such as the provision of information, informed consent, and legal redress.
- The Meaning of the AI Convention for Health and Patients
Lifecycle Regulation of General AI Systems
The definition of the proposed AI Convention is rather broad and includes most medical AI applications: “a machine-based system that is capable of informing or autonomously generating outputs such as predictions, recommendations or decisions using machine and/or human-based data and inputs and influencing the environment that it interacts with.” This means it covers applications like ChatGPT-assisted diagnosis, AI-based fall detection in elderly care facilities, AI-assisted surgery, and menstruation cycle trackers.
If the scope would be limited to uses in the public sector, it would still cover a fair amount of medical AI applications, such as automated disease surveillance, social benefits distribution, and most COVID-19 apps, plus many public health care systems. Moreover, the AI Convention applies to the entire lifecycle of AI: the design, development, and application of AI systems. This is especially important for adaptive AI systems that evolve over time.
This sets the Convention apart from the EU’s AI Act, which focuses on regulating the development phase, and only stipulates strict rules for “high-risk” AI medical devices.
Enforceable Rights for Patients
The AI Convention departs from the understanding that all human rights are potentially impacted by the growing use of AI systems, instead focusing on the specific rights to non-discrimination, privacy and data protection, transparency rights, and the right to redress for harm. It establishes these as enforceable rights — filling an often-criticized gap of other AI regulations.
Foremost, the Convention stipulates that all AI systems undergo a human rights assessment before deployment. Then, the Convention recognizes the right to non-discrimination, which is important for health equity — i.e., in cases of limited internet access in remote areas and insufficient digital skills to understand AI tools. The Convention protects privacy and data protection rights, setting high requirements for data protection, which is important for patients who often share intimate data with their doctor. The Convention also equips patients with the right to redress for harm, requiring robust frameworks for accountability and liability — a much-needed and currently absent solution for medical AI causing harm. The transparency rights put forward by the Convention contribute to the creation of a relationship of trust between patient and doctor and ensure informed consent to medical treatment.
Especially for health care — a sector that is mostly regulated locally — minimal human rights standards are important, as they give patients in all Member States the means to arm themselves against potentially harmful medical AI.
The Council of Europe Cares About Health
Generally, the AI Convention is promising for patients because of the Council of Europe’s long track record of protecting the health and human rights of patients. The Council has published many recommendations on medical-ethical issues aiming for stronger human rights protections in the medical field (i.e., health governance, organ donation, medical data protection). The Human Rights Court has enforced protection for patients against — inter alia — forced medical treatments, medical data breaches, and health discrimination. It has also established strong obligations for states to actively protect the health and patients’ rights of their citizens. Many states have changed their health practices and laws following Court orders, for example, for involuntary psychiatric commitment.
In short: health and patients’ rights benefit from a human-rights approach. The Council of Europe’s AI Convention, being the first binding fundamental rights instrument for AI negotiated at such a widespread scale, holds the potential of uplifting health and patients’ rights worldwide, when medical AI becomes common practice in health care.
