What you should know about the future EU AI Act (and why it matters to you)

Focus on people

Putting humans at the heart of artificial intelligence (AI). This is the goal set by the European Parliament. On 14 June, the plenary of the European Parliament approved the proposal for a regulation establishing harmonised rules on artificial intelligence. This milestone comes after the Internal Market Committee (IMCO) and the Civil Liberties Committee (LIBE) approved the proposal on 11 May. This approval sets the Parliament’s position for negotiations with the European Council. It will be the world’s first legislative to extensively regulate artificial intelligence.

The question is, what does the text include and how does it affect us? Parliamentary sources describe the proposal as a step forward in ensuring that AI systems are “safe, transparent, traceable, non-discriminatory and environmentally friendly”. Some of the main measures include:

• Risk-based approach: the level of obligations for provider’s depends on the level of risk AI may pose. Therefore, it is not specific technologies that are regulated, but their uses. Four levels of risk are established: unacceptable risk, high risk, limited risk and minimal risk;
• AI systems with an unacceptable level of risk will be prohibited. For example, those that deploy subliminal or intentionally manipulative techniques, exploit people’s vulnerabilities or are used to score individuals;
• Increase of prohibited “intrusive and discriminatory uses of AI”. As a general rule, “real-time” remote biometric identification systems in public access spaces will be prohibited. Biometric categorisation systems using sensitive characteristics (e.g. gender, race, ethnicity, citizenship status, religion, political orientation) will also be prohibited;
• Providers of general-purpose IA systems must assess and mitigate the risks posed by their models. This will involve an analysis of aspects such as impact on fundamental rights, health, security, safety, the environment, democracy and the rule of law; and
• Generative foundation models, such as Chat GPT, will have to meet additional transparency requirements, such as disclosing that the content has been generated by AI and publishing summaries of the copyrighted data used.

High-risk AI systems

The Parliament continues to consider as high risk those AI systems that are themselves a product regulated by European legislation and subject to conformity assessment in relation to their health or safety impact and risks (e.g. medical devices or medicinal products), as well as those AI systems that are a safety component of such a product.

High-risk systems are those that present a risk of harm to the health, safety or fundamental rights of humans or the environment. Two categories are established:

• Systems that are used in products to which EU product safety legislation applies, and
• Systems falling into eight specific areas and to be registered in a database at EU level. One of these areas is biometric identification and categorisation of natural persons.

With this definition, many AI systems used in the life sciences sector subject to conformity assessment by a notified body (medical devices) may be classified as high risk.

This classification entails (i) obligations for their placing on the market, such as having risk assessment and mitigation systems, ensuring clarity of information to the user or enhanced security measures, including an adequate level of human review and (ii) ongoing monitoring during their lifetime.

It is undeniable that AI will have a transformative effect on the management of people’s health. Precisely because of this impact, comprehensive regulation is essential to maximise its potential, while preserving the necessary guarantees for its functioning.