‘Black box’ AI models threaten transparency in financial services

Brussels-based NGO Finance Watch has warned that artificial intelligence (AI) is exposing EU consumers to growing risks in everyday financial services, from loan approvals to insurance pricing, and is calling for changes to EU rules before the bloc’s AI Act comes into force.

In a new report published on Tuesday, the group argues that while AI brings efficiency benefits, it also “entails major consumer risks that need to be properly mitigated by appropriate and robust regulations.”

The report outlines how AI systems are increasingly being used in key areas of retail finance, including credit assessments, insurance underwriting, robo-advice and chatbots but with little transparency or legal recourse for consumers when things go wrong.

Among the top risks highlighted are financial exclusion, price discrimination, mis-selling of unsuitable investment products, and the denial of legitimate insurance claims. These risks are amplified, the group says, by the “black box” nature of many AI models and the lack of sector-specific rules addressing AI in finance.

“The use of AI in the retail financial services market brings with it efficiency gains and cost reductions for financial institutions. For consumers, AI use may translate into speedier processes when concluding contracts. However, if not properly regulated and supervised, the use of AI tools in the consumer financial services market brings considerable risks,” it says.

Finance Watch criticises the narrow scope of the EU AI Act, adopted in 2024 and set to apply from August 2026. Under the law, only two financial use cases, credit scoring and pricing for life and health insurance, are currently deemed “high-risk” and subject to mandatory safeguards. The NGO calls for this to be expanded to cover all AI systems used in retail financial services.

“Not classifying the application of AI in the provision of all retail insurance products and bank accounts as high-risk renders the EU AI Act inconsistent with its goal of preventing AI systems from posing a risk to consumer access to essential private services and benefits,” the report says.

The group is also pushing for an EU-wide AI liability regime that would make it easier for consumers to seek redress. In February, the European Commission withdrew its proposed AI Liability Directive, citing competitiveness concerns, a move Finance Watch says leaves a major gap in consumer protection.

“Given AI systems’ lack of explainability, it is difficult, and in some cases impossible, for consumers to easily seek redress,” the NGO warns, calling for a reversal of the burden of proof to help individuals challenge unfair outcomes.

Finance Watch says the EU must act quickly to close these gaps. “Implementing the right regulation of AI and promoting its development and uptake in the EU go hand-in-hand and must be pursued in parallel; they must not be seen as being in opposition to each other.”