Oscar Health: documentation nearly 40% faster, claims escalations resolved 50% faster, and the first BAA with OpenAI

• Oscar brings AI to health insurance, reducing costs and improving patient care — OpenAI (customer story), 2024-04-01
• Oscar Claim Assistant Built On GPT-4 (Skeleton Trace, итеративные запросы, 100%/80% точности по категориям) — Oscar Tech (Medium, инженерный блог), 2023-10-19
• Harnessing OpenAI to Enhance the Healthcare Experience (документация <12 минут против 20+) — Oscar Health (корпоративный блог), n/a

Background

Oscar Health is a U.S. health insurance technology company that builds coverage around data and technology. Co-founder and CTO Mario Schlosser explains why language models became the company's inflection point: 'Language models were the first time we thought, you can translate the messiness of the real world into a clear digitized plan.' For an insurer that is no metaphor — its entire business consists of unstructured text: medical records, contracts, claim-processing logs, and correspondence with physicians.

Two institutional decisions set Oscar apart from most players. First — model-selection discipline: 'We benchmark all the major models on a regular basis, with proprietary datasets for healthcare-specific use cases,' says Nikhita Luthra, Senior Product Manager and AI R&D Lead. 'The OpenAI models consistently perform the best.' Second — the legal construct: Oscar became the first insurance company to sign a Business Associate Agreement (BAA) with OpenAI — the HIPAA-mandated contract that allows a third-party service to process protected health information. 'OpenAI has been a great partner for making sure the data is being used in a really responsible, compliant way,' Luthra notes.

The company is unusually open for its industry: the engineering team publishes technical deep-dives of its AI systems in the Oscar Tech blog on Medium (including a detailed post on the claims assistant's architecture, October 2023), and results in the corporate blog. This case is assembled from three such primary sources: the OpenAI case study (April 2024), the engineering write-up, and the company blog — a rare chance in healthcare AI to cross-check the marketing and technical layers of one story.

Problem

Administrative overhead is one of the main cost drivers in U.S. healthcare. Documenting a single conversation between a patient and the medical team takes a human more than 20 minutes; at an insurer's scale that is thousands of clinical hours spent typing text, and a direct contributor to nurse and physician burnout.

Understanding the 'life story' of an insurance claim is even harder. Millions of contractual variables participate in processing, and every claim leaves behind a Claim Trace — a detailed log of all decisions the system made along its journey. When a doctor asks about a claim ('why was it denied?', 'why this amount?'), Oscar's teams must walk those logs manually — slow, expensive work requiring rare simultaneous expertise in medicine, contracts, and internal systems.

The third layer is medical records. They are written in unstructured natural language, and for patients with the most complex conditions they run to 500 pages: finding the relevant fact is a needle-in-a-haystack problem. Here hides the systemic bias Luthra speaks about: the sicker the person, the longer and more unwieldy their record — and the worse the system serves precisely those who need help most.

Above it all sits the regulatory frame: HIPAA. Any solution that sends medical data to a third-party model is simply illegal without the BAA construct — the first filter that kills most 'quick pilots' in U.S. healthcare.

Solution

On the OpenAI API the company automated three loops, each with a primary source behind it.

Loop one — clinical documentation: drafts of care-conversation documentation and lab-result reviews. Time fell by nearly 40% — per the company blog, documenting one conversation now fits under 12 minutes versus 20+ for a human; nurses and clinicians shift to higher-order tasks.

Loop two — the claims assistant, whose engineering history is instructive (Oscar Tech, October 2023; authors Benjamin Baker, Evan Poe, Nazem Aldroubi). The naive approach — 'feed GPT-4 the whole Claim Trace' — hit context limits; stripping intermediate data helped, but the model got lost in complex cases. The working solution was the Skeleton Trace structure — a 'table of contents' of the trace showing only the module execution hierarchy without values: the model looks at the skeleton, decides which trace segment to request, and iteratively digs toward the answer; the system assesses the model's confidence and pulls additional data when in doubt. Results by task category: from 100% accuracy on several question classes to 80% on the hardest. In operation, the assistant cut the claims team's escalation resolution time by 50%, with accuracy on par with or better than human agents; the company expected to automate investigation of at least 4,000 tickets per month (48,000 by year end).

Loop three — medical records: finding relevant information to speed claims review, summarizing prior records to prep a provider for an encounter, and extracting data across many records for questions like 'which of these diabetic patients would be a good fit for continuous glucose monitoring'.

There is also a fourth, cultural loop — the 'knowledge flywheel': Oscar deliberately publishes its generative AI findings on its blog and social media. Schlosser's logic is pragmatic: 'People are all grappling with the same issues in healthcare. If we solve a problem first, we should tell others about it — they'll tell us what they solved, too, and that's a fantastic way of getting the flywheel spinning.' And the stated ambition reaches far beyond administrative routine: 'We don't just want to nibble around the edges of administrative use case simplification. In the next three to five years, we need to bring down the cost of seeing physicians and being in the hospital by a factor of 10. The only way to do that is to have a model front-and-center — not just scribing, but integrating into member-provider interactions.'

Organizationally, all of this is run by a dedicated AI Pod — a central team mandated to shepherd product, data science, and operations teams through applying AI. The philosophy starts from the business problem, not the technology: 'The most successful applications happen when we're able to break down a very complicated problem into bite-sized tasks,' says Luthra, adding a team detail: five of the Pod's six members are women, all in their twenties and thirties. Finally, the company plays the long game with the regulator: together with the White House, Oscar led a coalition of 37 of the biggest healthcare payers and providers on principles for responsible AI in healthcare. 'The biggest takeaway has been that the regulators in healthcare want AI to succeed, and they're incredibly excited. It's on us to keep up the transparent flow of information to the government,' says Schlosser.

Result

Per the OpenAI case study (April 2024): time spent documenting care conversations and reviewing lab results fell by nearly 40% (per the company blog — from 20+ minutes to under 12 per conversation); the claims assistant cut escalation resolution time by 50% with accuracy on par with or better than human agents; the expectation — automating investigation of at least 4,000 tickets per month (48,000 by year end). R&D experiments showed productivity gains of up to 90% in some cases. The engineering write-up adds granularity vendor cases usually lack: on simple scenarios the machine matched humans immediately, on complex ones it initially lagged, and after Skeleton Trace was introduced several question categories reached 100% accuracy, the hardest — 80%.

Framing: all figures are Oscar's self-report in an OpenAI case study (an interested party); 'up to 90%' is an R&D result, not production; 4,000 tickets/month was an expectation at publication time, not a confirmed fact; payout decisions remain with humans — AI accelerates finding and preparing information. The ambition to 'bring down the cost of seeing physicians by a factor of 10 in three to five years' is Schlosser's declaration of direction, not a commitment with a metric.

In our view, the case carries two systemic lessons beyond insurance. First: the sequence 'BAA first, models second' is the right order for any regulated industry; Oscar gained its head start not because it had access to different models, but because it was first to build the legal construct in which those models could lawfully touch real data. Second: the gap between 40% (production) and 90% (R&D) is the honest price of moving from demo to operations, and companies that publish both numbers with explicit labels deserve more trust than those reporting '90% everywhere'.

A third, engineering observation: the claims assistant's history shows that the main work of putting LLMs into legacy systems is not prompting but data-representation design. GPT-4 failed until the traces were repackaged into a Skeleton Trace with iterative detail retrieval — in effect, the team invented for its task what would later become the common agentic-RAG pattern. We would read this case precisely as a textbook on 'preparing data for the model', not as an advertisement for a particular API.

Lessons learned

1. In healthcare the legal construct comes first: the BAA with OpenAI separates a viable project from unacceptable risk; Oscar did it before any other insurer and gained a head start.
2. Regularly benchmarking models on proprietary datasets ('we benchmark all the major models on proprietary data') is the right substitute for choosing a model by marketing.
3. Separate R&D from production in reporting: Oscar honestly labels 90% as a research ceiling and ~40% as the working result — a rare and correct discipline.
4. The main work of LLMs in legacy systems is data representation, not prompts: GPT-4 failed on full claim traces until they were repackaged into a Skeleton Trace with iterative detail retrieval.
5. A centralized AI Pod mandated to shepherd other teams scales expertise without hiring ML engineers into every department, and decomposing hard problems into 'bite-sized tasks' is its core method.
6. The biggest wins hide in the most inconvenient data: 500-page records of complex patients turn from a barrier into an advantage — including the equity argument.
7. Public self-regulation (a 37-player coalition with the White House) is a way to shape future rules rather than wait for them.

Frequently asked questions