Seeking to rein in healthcare fraud, the US Centers for Medicare & Medicaid Services (CMS) is seeking explainable AI models that can identify patterns suggestive of malfeasance.
A chili cook-off usually involves stuffing one’s face with spicy food, but not at CMS, where the fraud-fighting AI competition has been dubbed the “Crushing Fraud Chili Cook-Off” despite no sign that stew is included, or sponsorship from an official chili cook-off body (yes, they exist).
Instead, the competition is seeking “market-based research” projects that make use of AI and ML models to “identify anomalous patterns within Medicare claims data, translate detected patterns into indicators of fraud, and propose scalable analytic and policy solutions to address the identified indicators.”
However, the project isn’t stopping there. “Pattern detection alone is not sufficient to determine, let alone prove, fraudulent behavior, especially in legal or enforcement contexts,” CMS said in the overview of the challenge, explaining that’s why it’s looking deeper than just identifying fraud patterns.
Instead, the cook-off is pushing participants to develop AI models that can “understand the underlying factors driving these anomalies” in order to “proactively flag similar fraud schemes across Medicare claims data and enhance the efficiency of program integrity efforts.”
Of course, since it’s asking for an explainable AI, CMS wants models that have transparent thought processes so it can understand how they’re arriving at their conclusions – a key part of determining whether the AI models are actually able to identify those underlying factors.
CMS is leaving it relatively open to participants to determine how best to meet the cook-off objectives, only noting that the first phase of the program involves submission of a white paper. The structure of that paper is rather specific, requiring how the explainable AI would work, how model outputs would be translated into actionable items, and how the system could be scaled “to detect systemic vulnerabilities.”
That’s a lot to ask from a simple white paper, but that’s all teams will be able to submit to prove they’re worthy of advancing to stage two of the cook-off, in which ten teams will be given access to a bunch of Medicare data to run their models against. Finalists will be given access to limited data sets from 2022-2024 that include paid hospice services, Medicare Part B charges, and medical equipment claims from a random subset of five percent of Medicare recipients.
Participants will not retain permanent access to the data, but other than that, CMS made no mention of how it intends to protect the personally identifiable information (PII) issued during phase two of the cook-off, nor did it respond to our questions.
That said, there’s definitely a need to address fraud in Medicaid and Medicare, which is the closest thing the US has to socialized medicine, the former being for the poor and the latter for the elderly. Despite being extensive, neither program covers all healthcare costs.
Both have been popular targets for fraudsters, with an estimated $100 billion lost to fraudulent claims each year. Health and Human Services secretary Robert F Kennedy Jr has made fighting that fraud one of his goals in between the elimination of proven vaccines and termination [PDF] of hundreds of research grants.
For those less interested in punishing their taste buds than they are developing anti-fraud AI models, the CMS cook-off is now open for registrations for the first round, with registrations closing September 19. Winners will be announced in mid-December, but don’t expect a cool prize or funding for further research.
“CMS will publicly recognize the ten finalists and the Chili Cook-Off winner for their innovative solutions on CMS social media channels,” the program page indicates. “Participants are welcome to leverage these innovations for any future contract bids that CMS announces.”
Not if CMS beats them to it first, naturally.
“Participation in the Chili Cook-Off requires teams to consent to granting CMS an irrevocable, paid-up, royalty-free nonexclusive worldwide license to reproduce, publish, post, link to, share, and display publicly the submission summary and abstracts,” an FAQ on the contest page indicates. While teams will retain other intellectual property rights to their submissions, CMS said it can use participant findings for its own work “to develop innovative changes to existing program integrity efforts.”
In other words, CMS may very well come up with its own chili recipe that looks suspiciously like the winner’s once the contest is over. After all, the secret’s in the sauce, and that’s what it appears CMS wants to get its hands on. ®