Get a summary of this article with your favorite AI
Key Highlights
- •Human-in-the-Loop Requirements mean AI can prepare appeal work, but accountable reviewers must see evidence and change final outcomes.
- •U.S. human oversight requirements vary by plan, decision, and party; state laws, federal rules, contracts, and liability can shape oversight.
- •State rules differ across utilization review, prior authorization, appeals, and claims; Florida has human-review rules, while workers’ comp remains separate.
- •Medicare Advantage requires individualized clinical review, while ERISA appeals require an independent review and qualified clinical input.
- •AI decision-making oversight works best when AI flags and recommends, while reviewers verify evidence, challenge outputs, and control submission.
- •Bill Matters builds evidence-backed appeals from EOB analysis, then keeps review, edits, approval, and submission with the responsible person.
This article is informational and does not replace legal counsel or clinical judgment.
If AI can identify a claim denial in seconds, should it also decide whether the appeal is ready to send?
No, it shouldn't which human in the loop requirements matter precisely because speed does not make an AI decision accountable. AI can handle much of the analysis and preparation, but the person responsible for the appeal still needs control over the final decision.
The regulatory landscape is moving in the same direction. States are now introducing AI-related requirements for claims, prior authorization, utilization review, and appeals, while Medicare Advantage and ERISA plans bring separate federal requirements. The obligation can change based on the state, plan type, decision involved, and who is making it.
So, what should you do when the law does not explicitly require a human checkpoint for an AI-assisted appeal?
Keep the checkpoint anyway. When Biz4Group LLC built Bill Matters, human review was not an afterthought but a priority from the start. The platform uses AI to handle the heavy work of analyzing denials, finding relevant evidence, and preparing appeals, while a person remains responsible for the final sign-off before an appeal reaches the payer.
That matters even more for medical billing companies,RCM heads and denial specialists managing high denial volumes. When hundreds of appeals move through a workflow, automation should reduce the workload without turning each appeal into an unattended decision.
This guide breaks down where human review is required, what meaningful human oversight requirements look like in practice, and how to put a real human checkpoint into AI-assisted claim appeals without giving the final decision to the machine.
What Are Human in the Loop Requirements for AI In Healthcare Claims and Appeals?
Human in the loop requirements define the point at which AI must stop acting on its own and a qualified person must take responsibility for the decision. In a claims or appeals workflow, that means the process needs a clear human checkpoint with enough control to question the AI output, act on the underlying evidence, and change the outcome when necessary.
That is the practical difference between human review of AI decisions and simply putting a person somewhere in an automated workflow.
What Makes a Human Checkpoint Meaningful?
The easiest way to test a workflow is to look at what the reviewer can actually do, not what the policy says the reviewer is supposed to do.
Use these five questions to pressure-test the checkpoint:
| Checkpoint | A real human checkpoint | Warning sign |
|---|---|---|
Can the reviewer see the evidence? | The reviewer can access the claim, denial, supporting documentation, and relevant payer requirements used to assess the AI output. | The reviewer receives only the AI's conclusion or a summary. |
Can the reviewer change the outcome? | The reviewer can edit, reject, approve, or escalate the recommendation. | Approval is the only available action. |
Can the reviewer make an independent judgment? | The workflow gives the reviewer enough information and capacity to assess the individual case. | Volume or turnaround pressure makes meaningful case-by-case review unrealistic. |
Can the reviewer override the AI? | An override is possible and recorded when the recommendation does not hold up. | The system discourages, hides, or fails to record overrides. |
Is responsibility traceable? | A named reviewer is tied to the final action and the decision history is preserved. | It is unclear who actually made or approved the decision. |
These checks matter because a human checkpoint is only useful when the person can change what the system would have done. A name attached to an AI-generated appeal does not create meaningful oversight by itself.
Consequently, could the reviewer have stopped the appeal, changed it, or sent it back because the evidence did not support the AI's recommendation?
If the answer is no, the workflow is still heavily dependent on automated decision-making. Now,
Is Human in the Loop Legally Required in the US?
Not universally. The U.S. does not have one federal rule that requires every payer, provider, billing company, or AI vendor to use the same human-review model. The applicable requirement depends on the decision being made, the organization making it, and the rules governing the plan or claim.
For AI in claims processing regulatory compliance, the current framework comes from three places:
- State insurance laws, particularly rules governing prior authorization and utilization review.
- Federal requirements, including rules affecting Medicare Advantage and ERISA-governed plans.
- Provider-side obligations, such as payer or client contracts and the liability that comes with submitting an AI-assisted appeal.
Therefore, there is no single definition of compliance that applies to every AI-assisted healthcare decision. The requirements change depending on the workflow and the parties involved.
Which Human in The Loop Laws Bind Payers, Providers, And AI Vendors?
The laws do not bind all three parties in the same way. Most of the direct statutory requirements in this area apply to payers and utilization review entities. Providers and medical billing companies face more limited state-specific requirements, while AI vendors can fall within the rules when they perform a regulated function for a payer.
Contracts and BAAs can create additional obligations, but those are separate from the statutory requirements. Here is the practical split:
| Party | Primary legal framework | What the requirements can cover |
|---|---|---|
Payers and utilization review entities | State insurance laws; Medicare Advantage and ERISA rules where applicable | Adverse determinations, utilization review, prior authorization, and certain appeals |
Providers and medical billing companies | Specific state statutes, plus payer and client contracts | Automated claim submission and other state-specific requirements |
AI vendors | Applicable statutes when performing regulated payer functions; BAAs and contracts | Responsibilities tied to the function the vendor performs and the terms of the relationship |
For practices and billing companies, that distinction matters when applying human oversight requirements for AI in medical billing. A rule governing automated claim submission does not automatically establish the requirements for an AI-assisted appeal.
Which US States Have Turned Human-in-the-Loop Requirements Into Law?
Human in the loop requirements are now written into healthcare laws across several states, but the rules do not all apply at the same point in the claims process.
Some control the initial adverse decision. Others require a human reviewer when the decision is appealed. A few regulate related automated actions such as downcoding or claim submission.
For teams evaluating human oversight requirements for AI denial management, that difference matters. A law that requires human review of the initial decision does not automatically create the same requirement for the appeal.
The human in-loop requirement laws below shows how the checkpoint is being defined across the country. The key details are the same ones that matter when you evaluate an AI workflow: who must review the decision, what decision they control, and whether the requirement continues into an appeal.
1. California- SB 1120 (Effective January 1, 2025)
A physician or qualified licensed health professional must make a medical-necessity decision when AI, an algorithm, or another software tool is used in utilization review. The system cannot independently deny, delay, or modify care and must consider the patient's individual clinical circumstances.
2. Illinois- HB 2472 (Effective January 1, 2025)
When a health plan or utilization review program uses an automated process for an adverse medical-necessity determination, only a clinical peer may make that determination. The law also carries that human-review requirement into the appeal, making Illinois one of the clearest examples of an appeal-specific checkpoint.
Illinois also enacted SB 3114, which will take effect from January 1, 2028. That law addresses downcoding rather than ordinary claim denials and requires physician involvement in specified downcoding decisions.
3. Texas- SB 815 (Effective September 1, 2025)
A utilization review agent cannot use an automated decision system to make an adverse determination, either wholly or partly. AI and other automated systems can still be used for administrative support and fraud detection.
4. Maryland- HB 820, Chapter 747 (Effective October 1, 2025)
When AI, an algorithm, or other software is used for utilization review, the system cannot replace the provider's role or independently deny, delay, or modify care. The law also requires the review process to account for individual clinical information and includes audit and performance-review requirements.
5. Nebraska- LB 77 (Effective January 1, 2026)
For specified prior-authorization adverse determinations, the decision must be made by a physician or, in specified circumstances, a clinical peer. Nebraska also has one of the clearest appeal protections: a physician who was not involved in the initial determination must review the appeal and consider the relevant clinical records and medical evidence.
6. Arizona- HB 2175 (Effective July 1, 2026)
Before an insurer can deny a claim or prior-authorization request involving medical necessity, a medical director must individually review the denial and exercise independent medical judgment. The statute does not specifically rely on the term AI, but it creates a human decision checkpoint regardless of whether the underlying recommendation came from an automated system.
7. Indiana- HEA 1271, Public Law 88 (Effective July 1, 2026)
Indiana addresses human review at two different points. A provider using an automated tool to submit a health benefits claim must have a person review the claim before submission.
The law also limits certain automated medical-necessity downcoding by insurers. These provisions matter for human in the loop AI for medical billing, but they are not a general AI appeal rule.
8. Washington- E2SSB 5395, Chapter 157 (Effective January 1, 2027)
Washington does not allow AI to be the sole means of denying, delaying, or modifying care based on medical necessity. A licensed physician or licensed health professional must review the clinical issues involved, and AI may not be used without human review to deny care on medical-necessity grounds. The law also requires individual clinical information, auditability, and periodic review of AI performance.
9. Iowa- HF 2635 (Effective July 1, 2026)
Iowa allows AI to assist with prior-authorization review, but it cannot be the sole basis for a medical-necessity denial, delay, or downgrade. Its appeal provisions add a separate human checkpoint by requiring review by a qualified person or clinical peer who was not involved in the initial decision.
For AI appeals management, that separation matters: the person reviewing the appeal is not simply carrying forward the original automated decision.
10. Alabama- SB 63 (Effective October 1, 2026)
Alabama requires covered prior-authorization decisions involving medical necessity to use patient-specific clinical information and places the decision with a licensed physician or qualified health professional. The law also adds AI disclosure, certification, and system-review requirements.
11. Utah- SB 319 (Effective January 1, 2027)
Utah requires independent medical judgment in specified preauthorization decisions and adds disclosure requirements when AI is used in the review process. Its appeal provisions also require physician review for specified physician-requested medical-necessity appeals.
That makes Utah relevant to AI decision-making oversight because the requirement is not simply about placing a human somewhere in the process; the reviewer must exercise independent medical judgment.
12. Colorado- HB 26-1139 (Effective January 1, 2027)
Colorado allows AI to support utilization review, but a denial based in whole or in part on medical necessity cannot be issued solely from AI output. The denial must be reviewed by a licensed clinician, physician, or other competent regulated professional. The law also requires human-oversight and audit processes around the AI system.
13. Georgia- SB 444 (Effective January 1, 2027)
Georgia requires human involvement when AI is used in utilization review. An adverse determination cannot be issued from AI alone; a natural person must conduct the review, and a clinical peer must participate, with the AI system unable to override that clinical judgment.
14. Minnesota- HF 4188 (Human-review provision effective January 1, 2027)
Minnesota requires a physician to review and make an adverse clinical determination in utilization review. The enacted provision also prohibits a utilization review organization from using automated processing alone without review by an appropriate health professional.
Now, few things to remember:
1. What These State Laws Mean for AI-Assisted Appeals
The state landscape does not create one uniform rule. It creates different levels of human control around automated healthcare decisions.
The most important differences are:
- Who must review: physician, clinical peer, medical director, or another qualified professional.
- What they control: utilization review, prior authorization, claims, downcoding, or another defined decision.
- Whether the appeal is covered: Illinois, Nebraska, and Iowa provide particularly clear appeal-specific protections, while Utah also addresses specified physician-requested appeals.
- Whether AI can be the sole basis: several states explicitly prohibit that model.
- Whether independent judgment is required: Arizona and Utah make this especially clear.
One distinction is worth keeping in mind when applying these laws: prior authorization, utilization review, and downcoding are not the same decision. A human-review requirement attached to one does not automatically govern the others.
2. Workers’ Compensation Follows a Separate Regulatory Framework
Workers' compensation follows separate state-specific rules and claims operate under state workers’ compensation statutes, carrier rules, and applicable fee schedules, rather than the same health-plan framework used for commercial medical claims.
Some of the laws discussed above also expressly exclude workers’ compensation from their scope.
Illinois and Alabama, for example, exclude workers’ compensation from specified downcoding provisions. Tennessee takes a separate approach for utilization review, requiring a physician of the same or similar specialty, licensed in Tennessee, to make the relevant determination.
For a workers’ compensation billing specialist, the practical rule is simple: do not assume a human-review requirement that applies to a commercial health plan automatically applies to a comp claim. Check the applicable workers’ compensation statute and carrier rules for that jurisdiction.
Does Your AI Workflow Know Who’s Accountable?
Bill Matters keeps denial automation inside a review-first workflow with clear human decision control
What Human-Review Requirements Apply to Medicare Advantage and ERISA Plans?
Medicare Advantage and ERISA plans follow separate federal human-review requirements. These are not AI-specific laws. They are existing federal claims and appeals rules that still apply when AI is used in the decision process.
Consequently, a state without a specific AI human-review law can still have claims subject to federal human-review requirements when the plan falls under Medicare Advantage or ERISA.
1. Medicare Advantage: Individualized Clinical Review
Medicare Advantage organizations must base medical-necessity decisions on the individual enrollee's circumstances, including relevant medical history, physician recommendations, and clinical notes. CMS does not permit a population-level algorithm to replace that individualized assessment.
When an organization expects to issue an adverse medical-necessity determination, the decision must be reviewed by a physician or another appropriate healthcare professional with relevant expertise before it is issued.
The reconsideration has a separate safeguard: when the initial determination involved medical necessity, the reconsideration must be performed by a physician who was not involved in the initial decision.
For human in the loop AI claims denial requirements, the practical rule is clear: AI may support the analysis, but the required clinical review cannot be replaced by the model.
2. ERISA: Independent Appeal Review
ERISA applies a different control. For a covered group health plan, an appeal must receive a full and fair review without deference to the original adverse determination. The person reviewing the appeal cannot be the original decision-maker or that person's subordinate.
When the appeal involves medical judgment, the plan must consult a healthcare professional with appropriate training and experience in the relevant field. That professional also cannot be the person consulted for the original determination or that person's subordinate.
That gives human in the loop requirements for medical claim appeals a specific federal form under ERISA:
- The appeal gets a fresh review.
- The original decision-maker cannot decide the appeal.
- Medical-judgment appeals require qualified clinical input.
For AI decision-making oversight, the important point is that ERISA's independent-review requirements continue to apply even when the original decision or appeal preparation involves automated tools.
What Human-Review Rules Apply to Florida AI Claim Denials?
Florida does not have an enacted AI-specific law requiring human review solely because AI was used in a claim decision. But Florida already requires human review in specific medical-necessity and HMO appeal processes.
The 2026 HB 527 proposal would have created a broader AI-specific requirement, but it did not become law. The House passed it 108–0 on March 5, 2026, and the Senate Rules Committee killed it on March 13. Its companion SB 202 also died on March 13.
The current requirements come from the applicable Florida claim or HMO rules. For self-funded employer plans and Medicare Advantage, the federal requirements apply instead.
1. Florida Claims Regulatory Frameworks
| Florida workflow | Current Law | Current human-review requirement |
|---|---|---|
Fla. Stat. §627.6141 | A claimant or provider can appeal to the insurer's licensed physician responsible for medical-necessity reviews or a physician in the plan's peer-review group. The physician must respond within 15 business days. | |
Fla. Stat. §641.51(4) | An adverse determination involving a service provided by a physician must be rendered by a properly licensed physician. The notice must identify that physician and explain the appeal process. | |
Fla. Stat. §641.511(4) | The internal review panel must have a majority of members who were not involved in the initial decision, and a majority of the reviewers must be providers with appropriate expertise. | |
Expedited HMO review | Fla. Stat. §641.511(6) | An appropriate clinical peer must conduct the review, and that reviewer cannot have participated in the initial adverse determination. |
For human in the loop AI for medical billing, these rules matter because the human checkpoint can already be part of the claim or appeal process even without an AI-specific Florida statute.
2. What HB 527 Would Have Added
HB 527 would have created a separate AI-specific requirement for insurance claim decisions. The proposed framework would have:
- Required claim-denial and payment-reduction decisions to be made by a qualified human professional.
- Prohibited AI, machine learning, or an algorithm from being the sole basis for adjusting or denying a claim.
- Required the human reviewer to independently analyze the claim and review the AI-generated output.
- Required records identifying the human decision-maker, review activity, and information used in the decision.
- Allowed the Office of Insurance Regulation to conduct compliance examinations and investigations.
Because HB 527 did not pass, those AI-specific requirements are not current Florida law.
3. What Orlando Teams Need to Identify?
The first step is to classify the case:
- Plan type: commercial, self-funded employer, Medicare Advantage, or workers' compensation.
- Decision type: claim denial, prior authorization, utilization review, payment reduction, or appeal.
- Review requirement: physician, clinical peer, qualified professional, or independent reviewer.
That classification determines which human oversight for automated medical claim decisions applies before the AI-assisted workflow is allowed to move the case forward.
Why Should AI Flag and Recommend Instead of Auto-Deciding Claim Appeals?
AI can do the heavy analysis behind an appeal. It can interpret the denial, trace the reason back to the claim, gather relevant evidence, spot gaps, and prepare the argument. The decision about whether that appeal is accurate, supported, and ready to submit still belongs to an accountable person.
AI's Role in the Appeal
The boundary becomes clearer when each side has a defined job:
| AI prepares | Assigned appeal reviewer decides |
|---|---|
Interprets the EOB and denial reason | Whether the denial was understood correctly |
Connects the denial to claim lines, codes, and supporting records | Whether those records support the appeal |
Retrieves relevant documentation and payer requirements | Whether the evidence is relevant and sufficient |
Drafts the appeal argument | Whether the argument accurately reflects the record |
Identifies missing information or documentation | Whether those gaps need to be resolved before submission |
Flags filing deadlines and required items | Whether the appeal packet is complete and ready to submit |
That is the practical boundary behind human in the loop requirements for AI claim appeals: AI prepares the case, while an accountable person decides what goes out.
Bill Matters is built around this boundary. After analyzing the EOB and denial details, it connects the denial reason to the underlying claim, identifies evidence that can support the appeal, and builds an evidence-backed draft. It can also surface gaps and suggest what additional records or supporting material the appeal packet may need.
What Should a Human Review Workflow Look Like When AI Drafts Claim Appeals?
A workable workflow should make three things obvious: what AI prepares, who checks it, and who has authority to release the appeal.
| Stage | Responsible role | What happens |
|---|---|---|
Flag | AI system | Classifies the denial, analyzes the EOB, connects the denial to claim details, retrieves supporting evidence, identifies gaps, and surfaces filing requirements. |
Review | Assigned appeal reviewer | Verifies the source records, codes, denial reason, payer requirements, supporting evidence, and AI-generated appeal. The reviewer can accept, edit, reject, or request more information. |
Decide | Authorized appeal owner | Makes the final submission decision. The appeal is approved, held, escalated, or rejected based on the completed review. |
Record | Appeal workflow system | Captures who reviewed the case, who made the final decision, what changed, when actions occurred, and whether the AI recommendation was overridden. |
These are responsibilities, not necessarily four different people. A small practice may have one person perform the review and final approval. A larger medical billing company or RCM operation may assign those responsibilities to different team members. What matters is that the reviewer and decision owner are identifiable.
2. Reviewer Decision Rights
The reviewer should not be limited to approving an AI-generated letter. The workflow needs explicit authority to challenge the output.
The reviewer should be able to:
- Edit an incorrect or incomplete argument.
- Reject an unsupported AI recommendation.
- Request evidence when the current packet does not support the appeal.
- Escalate cases that require clinical, coding, contractual, or legal judgment.
- Hold submission when a filing requirement or deadline issue remains unresolved.
- Approve submission only when the appeal meets the organization’s review standard.
This is what turns human review into an actual control rather than a final click before submission.
3. What the System Should Record?
A meaningful review should leave evidence behind. The audit record should show:
- Appeal and claim identifiers
- Denial and EOB information used in the analysis
- Evidence presented to the reviewer
- AI-generated recommendations and appeal draft
- Reviewer identity and review timestamp
- Edits, rejected recommendations, and overrides
- Additional evidence requested or added
- Final decision and submission status
The record matters because the organization should be able to reconstruct how the appeal moved from AI analysis to human approval. NIST guidance similarly emphasizes documenting human oversight, overrides, downstream actions, and accountable decisions in AI systems.
We built this workflow into Bill Matters. After the AI builds the appeal and identifies the supporting evidence, the appeal stays with the reviewer. It does not send it to the payer. The reviewer evaluates the evidence, changes the draft where needed, adds missing support, and decides whether the packet is ready for submission. The workflow also leaves a record of the reviewer’s identity and review timestamp.
Ready to Keep AI in Its Lane?
Let Bill Matters handle the appeal heavy lifting while your team keeps the final say
Where Human-in-the-Loop Requirements Ultimately Stand
As AI takes on more of the work behind claim appeals, the decision still needs a clear human owner. Bill Matters was built around that principle, keeping the reviewer in control of the appeal rather than allowing automation to make the final call.
For doctor practices, practice owners, medical billing companies, and RCM teams, the goal is not to slow automation down. It is to make sure automation does not remove accountability from the process. AI can support the work at scale, but the person responsible for the claim must still have the authority to question the output, change it, and decide whether it should move forward.
