When AI influences healthcare, who carries the risk?

AI is used across healthcare in very different ways, from administrative support and medical transcription to clinical decision support and diagnostics. Its risk profile depends on its function, who uses it and how directly its output influences patient care.  

It also matters whether a business has built its own model or uses a third-party tool. Who owns the underlying technology and data? And how do contracts divide responsibility between developers and healthcare providers? 

For brokers, getting beneath the “AI” label is the starting point. A tool supporting a clinician is a very different proposition from one making decisions more autonomously. 

Keeping a human in the loop can provide important clinical oversight, but it does not remove every risk. Staff still need to understand the technology, know its limitations and have a clear process for challenging or escalating unexpected outputs. 

That becomes particularly important when AI starts influencing diagnosis or treatment. If something goes wrong, where does responsibility sit between the clinician, healthcare provider and technology company? 

Training data is another key consideration. Its relevance, breadth and accuracy can influence how reliably a model performs for its intended users. Bias and model drift can also emerge over time, making ongoing monitoring as important as the controls in place at launch.  

For brokers, there are some practical questions worth asking: 

• What does the AI tool do and who uses it? 

• Is it an in-house or third-party model? 

• Where does human oversight sit? 

• How are errors, bias and model drift monitored? 

• What happens when a patient needs to be escalated? 

The answers can reveal a lot about both the exposure and how well it’s being managed. 

AI is challenging traditional ideas of responsibility in healthcare. As technology becomes more involved in patient care, liability can go beyond healthcare providers to include technology companies and other parties involved in developing or deploying the tool.   

That can create challenges for traditional insurance. Technology and cyber policies have historically focused on financial loss, while products liability cover was designed for physical products rather than software-powered healthcare tools.  

So bodily injury cover alone doesn’t necessarily tell the full story. Brokers may need to look at whether professional liability extends to injury caused by technology or products, and whether software is actually captured within the relevant policy wording. 

Regulation may not always develop as quickly as the technology, making governance, risk management and insurance important parts of scaling AI safely.  

Clear documentation, due diligence, staff training and ongoing monitoring can help businesses show how AI is being used and how its risks are managed. For insurers, that can also provide a much clearer picture of the exposure.