Clifford Chance

Artificial Intelligence (AI) is fuelling discussions on regulation, appropriate oversight, and ethical use. For the healthcare industry, the use of AI has at its core a range of social policy issues that intimately relate to individuals' medical treatment and ultimately, wellbeing, and new U.S. Government guidance will have a significant impact on health insurers.

In the most recent issuance of AI guidance by the U.S. government in the wake of the "Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence" (E.O. 14110), issued on October 30, 2023, the Department of Health and Human Services (HHS) Centers for Medicare and Medicaid Services (CMS) has provided pointed AI direction in recently issued guidance  "Frequently Asked Questions related to Coverage Criteria and Utilization Management Requirements in CMS Final Rule (CMS-4201-F)".

CMS Guidance

The CMS Guidance illustrates how government entities will be able to directly influence private sector usage of AI in the U.S. health insurance market. It also highlights that HHS will not sit idly by as AI issues in health insurance begin to manifest. The CMS Guidance comes amidst a number of pending class action suits against major US health insurers challenging the appropriate use and dependence on AI technology to determine coverage eligibility under Medicare plans.

The CMS Guidance issued on February 6, 2024, which takes the form of FAQs, mandates that health insurance companies that offer Medicare Advantage plans, Medicare Prescription Drug Benefit programs, and other Medicare services (Medicare Programs) align their use of algorithms and AI in patient coverage decisions with established criteria for medical necessity, taking into account the unique medical history and needs of each individual patient. The CMS Guidance emphasizes that healthcare coverage decisions for Medicare Programs may not be based solely on broad data sets or predictive models; such decisions must be based on individual assessments with human oversight.  Health insurers administering Medicare Programs must adhere to legal non-discrimination standards, and exclusive dependence on AI and algorithms does not meet this criterion. The practical implication for health insurers using AI in Medicare Program claims decision processes is that they need to be able to substantiate that AI technology is not used as the sole basis for making a coverage determination.

Takeaways

In the wake of the CMS Guidance, health insurers should rigorously assess the extent to which AI solutions, including those provided by third parties, are embedded within their Medicare claims operations. It is critical that any use of AI and other algorithmic technology supports, rather than replaces, personalized care decisions for Medicare Programs. An evaluation is necessary to understand the specific functions these technologies perform, and the level of human oversight. The CMS Guidance will require insurers to not just fully understand and carefully use AI solutions going forward in Medicare claims determinations, but also necessitates a lookback at existing AI deployments.   

As health insurers assess the use of AI in claims processing for Medicare Programs, important considerations include:

• Ensure Human Oversight: Health insurers should establish definitive protocols that guarantee human intervention in AI-driven decision-making. Specifically, insurers should implement checkpoints at critical junctures where human professionals review the decisions suggested by AI or algorithms. This step is crucial, particularly in complex cases or instances where AI-generated recommendations do not align with established care guidelines, ensuring that every decision adheres to the highest standards of personalized care.
• Conduct a Detailed AI Audit: Initiate a thorough audit of current AI implementations within Medicare claims processing. This includes identifying all AI tools and algorithms in use, especially those developed or managed by third-party vendors, to ensure full visibility into how AI is integrated into operations.
• Assess AI Impact on Decision-making: Evaluate how AI technologies influence decision-making processes, particularly related to claims and appeals, and closely examine how AI systems make decisions about patient claims and care. Specifically, insurers should verify that these AI-driven decisions consider the unique health needs, preferences, and circumstances of each Medicare beneficiary, rather than applying a one-size-fits-all approach. This ensures that the AI technology is used as a tool to support and enhance personalized care.
• Review and Update Training Data: Regularly assess and update the datasets used to train AI systems to ensure they are comprehensive, unbiased, and reflective of the diverse needs of Medicare Program members.
• Enhance Transparency and Explainability: Work towards making AI decision processes more transparent and understandable to both internal stakeholders and Medicare beneficiaries. This could involve developing explanations for AI decisions that are easily comprehensible.
• Monitor Compliance with CMS Guidance: Continuously monitor AI applications to ensure they comply with the latest CMS guidance and regulations regarding AI use.
• Implement Continuous Learning and Improvement: Create mechanisms for continuous learning and improvement of AI systems based on real-world outcomes, feedback from healthcare professionals, and Medicare beneficiaries' experiences.
• Engage with Regulators and Stakeholders: Maintain open lines of communication with regulators, healthcare providers, and members to gather insights on the effectiveness and fairness of AI applications in Medicare claims processing.
• Risk Assessment and Mitigation: Perform regular risk assessments of AI tools to identify potential biases or errors that could negatively impact Medicare beneficiaries. Develop strategies to mitigate these risks, ensuring equitable and accurate claim determinations.
• Educate and Train Staff: Provide ongoing education and training for staff on the ethical use of AI, the importance of human oversight, and how to effectively integrate AI tools into their workflows while prioritizing personalized care.

The CMS Guidance was not issued in isolation and further direction from CMS is likely to come as health insurers' varied AI use cases are further scrutinized. Importantly, the CMS Guidance emphasizes two key points in the continued AI discourse: (1) health insurers will be directly implicated as HHS begins to flex its regulatory authority implementing the Executive Order, and (2) the initial AI rhetoric of 2023 seems to be giving way to a rapidly maturing approach to handling the practical realities (versus speculative issues) posed by existing AI deployments in the healthcare sector.