Public benchmark · rcm_pro

The long-horizon revenue-cycle benchmark.

Recorded on real, de-identified claims. Each claim is a stateful, multi-stage workflow — eligibility → coding → 837P → denial rework → payment → audit.

150

Logged rollouts

18.7%

Full-resolution rate

0.34

Avg reward

4.0%

Clean first-pass

Leaderboard

Headline metric is full-resolution rate (pass@1): the claim closed paid with exact-match coding and survived any post-payment audit.

Model / policyResolution rate (pass@1)

1 GPT-5.5 Best openai 33.3%

          pass@3 40.0%avg reward 0.37coding 57.0%41 tool calls$0.50/run
        

2 GPT-5.4 openai 26.7%

          pass@3 40.0%avg reward 0.37coding 53.9%65 tool calls$0.83/run
        

3 Claude Opus 4.8 bedrock 23.3%

          pass@3 40.0%avg reward 0.33coding 50.9%70 tool calls$14.60/run
        

4 Qwen3-32B (thinking) bedrock 6.7%

          pass@3 10.0%avg reward 0.29coding 37.5%32 tool calls$0.06/run
        

5 Qwen3-235B (non-thinking) bedrock 3.3%

          pass@3 10.0%avg reward 0.32coding 47.7%62 tool calls$0.16/run
        

Tasks

10 claims spanning 5 easy · 3 medium · 2 hard.

hard 837P

Claim #403065

Professional (837P / CMS-1500) claim — code it end-to-end from the chart and resolve to payment.

clean10_benchmark + de-identified sample export

hard 837P

Claim #475902

Professional (837P / CMS-1500) claim — code it end-to-end from the chart and resolve to payment.

clean10_benchmark + de-identified sample export

medium 837P

Claim #383348

Professional (837P / CMS-1500) claim — code it end-to-end from the chart and resolve to payment.

clean10_benchmark + de-identified sample export

medium 837P

Claim #578284

Professional (837P / CMS-1500) claim — code it end-to-end from the chart and resolve to payment.

clean10_benchmark + de-identified sample export

medium 837P

Claim #1530447

Professional (837P / CMS-1500) claim — code it end-to-end from the chart and resolve to payment.

clean10_benchmark + de-identified sample export

easy 837P

Claim #56095

Professional (837P / CMS-1500) claim — code it end-to-end from the chart and resolve to payment.

clean10_benchmark + de-identified sample export

easy 837P

Claim #72067

Professional (837P / CMS-1500) claim — code it end-to-end from the chart and resolve to payment.

clean10_benchmark + de-identified sample export

easy 837P

Claim #73022

Professional (837P / CMS-1500) claim — code it end-to-end from the chart and resolve to payment.

clean10_benchmark + de-identified sample export

easy 837P

Claim #78489

Professional (837P / CMS-1500) claim — code it end-to-end from the chart and resolve to payment.

clean10_benchmark + de-identified sample export

easy 837P

Claim #95187

Professional (837P / CMS-1500) claim — code it end-to-end from the chart and resolve to payment.

clean10_benchmark + de-identified sample export