The financial services industry needs to be aware of bias when incorporating AI and machine learning (ML) models into their systems, according to an analyst at the Bank of England.

In a recent blogpost, Kathleen Blake warned that “AI may create or amplify financial stability and monetary stability risks.”

She said that AI and ML models can be biased by design because of the data used to train them and the structure of the underlying model.

Blake added that these biases can lead to discriminatory decisions in industries including banking and insurance.

Blake gave the example of a healthcare algorithm which was trained on cost data to predict patients’ health risk score. It was found to demonstrate bias in underrating the severity of Black patients’ health conditions relative to their white counterparts, leading to under-provision of health care to Black patients.

Blake said the use of AI is set to grow by three and a half times over the next three years and if firms use opaque or “black box” models, it would be difficult to predict how these models could affect markets.

Blake wrote: “Issues of fairness are cause for concern alone by some, but it might also be the case that they can exacerbate channels of financial stability risk since trust is key for financial stability. In periods of low trust or high panic, financial firms see increases in financial instability which can produce a spectrum of outcomes such as market instability or bank runs.”

She concluded that as firms continue to adopt AI, central banks will have to consider the risks around bias and other ethical issues and look at how to manage these risks.

Share Story: