The European Parliament recently approved an initial draft proposal for the regulation of ethical artificial intelligence (AI) and the first official draft of the proposal is expected in late April 2021. The proposal targets high-risk AI in the Healthcare and MedTech sectors in particular, but also sets standards for all AI products and applications. The scope of application of the proposal is very broad in that it covers all uses of AI products in the EU regardless of the origin or place of establishment of the developer or owner of the AI. It also regulates not only the developers of AI, but also the deployers and users of those AI products and applications.

What types of technology will be regulated?

The Regulation will likely apply to “artificial intelligence”, “robotics” and “related technologies”, including software, algorithms and data used or produced by such technologies, developed, deployed or used in the Union.

Each of artificial intelligence, robotics and related technologies are defined terms and broadly cover: AI software/hardware systems (artificial intelligence), physical machines with AI capability (robotics), and other technologies such as those capable of detecting biometric, genetic or other data (related technologies). The Healthcare sector is a key target for these proposed laws and medical devices deploying this technology will likely fall within scope.

How will it affect medical devices in particular?

The EU clearly has the MedTech sector in its sights with this proposal and high risk AI applications, in particular, will feel the full force of this regulation. It is expected that healthcare (including the MedTech industry) will be designated a high-risk sector and medical treatments and procedures including the use of MedTech will be designated high-risk uses or purposes. The practical effect of this is that medical device operators deploying AI can expect to be working against an assumption that their use of AI is high risk and subject to compliance.

High risk is to be determined by a Member State supervisory authority following a risk assessment based on objective criteria such as the specific use or purpose of the AI, the sector where they are developed, deployed or used and the severity of the possible injury or harm caused. So, those wishing to remain outside the supervisory authority’s remit will need to be able to demonstrate to the supervisory authority by reference to these broad criteria that its use of AI in a MedTech device cannot objectively be deemed high risk.

If, as may be likely, the MedTech owner’s deployment of AI is deemed high risk then it will need to proceed through a compliance assessment which will include (this is a non-exhaustive list):

• Guaranteeing full human oversight at any time, including in a manner that allows full human control to be regained when needed, including through the altering or halting the use of AI in the medical device. If not already part of the medical device this could amount to an obligation on medical device owners/operators to build into the device a remote access and control for operators to ensure control can be exercised by a human at any point

• An assurance of compliance with minimum cybersecurity baselines proportionate to identified risk and one that prevents any technical vulnerabilities from being exploited for malicious or unlawful purposes. Medical device owners/operators will need to demonstrate how resilient their devices are in relation to cyberattacks, and

• Proof that the device will operate in an unbiased manner and that it will not discriminate on grounds such as race, gender, sexual orientation, pregnancy, national minority, ethnicity or social origin, civil or economic status or criminal record. For a medical device that is deployed in identifying cancers this could mean demonstrating that the training and test data sets were broad enough so as not to give rise to poor outcomes for patients based on e.g. gender or race.

One of the key issues arising for medical device manufacturers is how this whole regulatory framework will work with the existing medical device framework. Obviously medical device manufacturers are already subject to stringent requirements and conformity assessment procedures under the MDD/MDR and IVDD/IVDR. It is currently unclear how the regulation of AI is going to fit into and within this existing framework and this will be clearly be a challenging issue for the European legislature. With draft legislation expected on 21 April 2021 it will be very interesting to see how this issue is approached given the known focus on the Healthcare industry.

Risk assessment and supervisory authorities

The proposal envisages mandatory compliance assessments for all devices that are found to be high risk and voluntary certificates of ethical compliance for all other AI. The process of certification prior to market launch is to be carried out locally at Member State level by supervisory authorities, which is very similar to the current regulation of data protection under the GDPR. It is proposed that an overarching group of supervisory authorities could meet at EU level with the Commission to oversee the operation of the certification and monitoring of AI.

Change in Liability Law

In this regulation proposal and a companion proposal for updating the laws on civil liability regarding the use of AI, there is a significant change proposed to the current law on product liability in the EU which will significantly impact those operating in the medical device supply chain. Currently manufactures shoulder almost all risk with regard to product liability under EU law. If this proposal is accepted manufacturers as well as those deploying AI in medical devices will be liable to users of the devices for loss, damage, or injury they suffer. This shift in responsibility for risk is seismic and will impact all those operating in the medical device field, including insurers.

Redress and conclusion

Any natural or legal person shall have the right to seek redress for injury or harm caused by operators of high risk AI, which will more than likely include medical devices. as a result of breaches of Union law and the obligations set out in the proposal. This early draft of this very important law will prove challenging for those developing and deploying high-risk AI. The proposed introduction of a new market certification regime for high risk AI applications in technology like medical devices would add to the existing significant EU regulatory burden on manufacturers and owners of medical devices. The expected change to the liability regime for products like medical devices deploying AI will be of greater interest and concern to the medical device sector as a whole. Those previously comfortably beyond the reach of mandatory product liability law may now find themselves at the centre of it. Watch this space for updates.

For more information, contact a member of our Life Sciences or Intellectual Property teams.

The content of this article is provided for information purposes only and does not constitute legal or other advice.