Clinical AI at Scale: What 271 Million Queries Reveal
Interest in artificial intelligence (AI) for clinical care has grown rapidly. Yet empirical evidence on who uses these tools, how often, and for what purpose has remained limited. Earlier estimates came from self-report surveys or website traffic data. These sources could not separate experimentation from routine use, and they said nothing about what clinicians were actually asking.
A new cross-sectional study in JAMA addresses this gap. It analyzes query logs from OpenEvidence, the most commonly used AI-enabled clinical information platform in the US, from January 1, 2024, through June 30, 2026. The authors argue that the findings point to a basic change in how clinicians access medical knowledge.
Study Design
The Platform and Population
OpenEvidence uses large language models (LLMs) to answer free-text clinical questions, with in-line citations to peer-reviewed literature and professional society guidelines. Access is free and limited to clinicians with a verified National Provider Identifier. The investigators linked users to national provider records. They then compared physician use with the American Medical Association’s active workforce data, using a “use ratio”: a group’s share of queries divided by its share of the workforce.
Classifying Query Content
The researchers drew a stratified random sample of 38.8 million queries, and an LLM classified each one by clinical purpose and medical topic. In a blinded validation, the model agreed with clinician annotators at least as often as clinicians agreed with each other: 73.6% vs 66.3% for primary purpose.
Key Findings
Rapid and Sustained Adoption
A total of 952,008 clinicians submitted 271 million queries. Of these clinicians, 556,200 were physicians, who accounted for 63.5% of query volume.
- Monthly active users rose from 355 to 703,265.
- Monthly queries grew from about 6,000 to 27.8 million.
- Month-over-month retention rose from about 70% to 90%, and by June 2026, 92.5% of active users were returning users.
- Only 9.6% of clinicians used the platform on a single day and never returned.
Nurse practitioners and physician assistants used the platform most intensively, at 44.5 queries per active user-month each. Physicians followed at 37.1 and pharmacists at 29.3.
A Generational Divide
Career stage produced the sharpest contrasts.
- Early-career physicians: those within a decade of graduation made up 24.3% of the workforce but submitted 61.8% of queries (use ratio, 2.54).
- Residents and fellows: they submitted queries at 1.84 times their workforce share.
- Senior physicians: those 20 or more years from graduation made up 53.0% of the workforce but submitted only 16.5% of queries (use ratio, 0.31).
The authors connect this pattern to what has been described as an “inversion of expertise,” in which learners may be more fluent with new technologies than their supervisors.
Specialty Variation
Use was highest in three specialties:
- oncology and hematology (use ratio, 2.00)
- internal medicine (1.88)
- family medicine (1.80)
Use was lowest in procedural and diagnostic fields:
- ophthalmology (0.23)
- radiology (0.24)
- orthopedics (0.24)
- anesthesiology (0.37)
What Clinicians Are Asking
Treatment, Not Diagnosis
Clinical decision-making made up 51.9% of physician queries. The other categories were:
- clinical knowledge: 37.2%
- evidence: 7.0%
- administrative tasks: 3.0%
- medical education: 0.9%
The single largest purpose was treatment and management (33.1%), followed by medication information (14.1%). Diagnosis accounted for only 13.9%. Pharmacists focused mainly on medication information (32.8%) and treatment (32.0%).
Specialty-Specific Priorities
Query purposes tracked each specialty’s clinical priorities.
- Psychiatrists asked medication questions at twice the overall physician rate (29.1% vs 14.1%).
- Oncologists most often asked about specific trials.
- Gastroenterologists and cardiologists asked about guidelines most often.
-
Crossing Specialty Boundaries
Physicians frequently asked about topics outside their own specialty.
- Surgeons directed 70% of queries to nonsurgical topics.
- Anesthesiologists directed 73% to nonanesthesiology topics.
- Even oncologists, the most self-contained group, directed 35% of queries elsewhere.
Out-of-specialty queries leaned toward basic disease and medication information. Within-specialty queries more often sought literature and guidelines.
Implications for Practice and Policy
An Evaluation Mismatch
Earlier evaluations of clinical AI have focused on diagnostic reasoning and licensing-examination benchmarks, where LLMs now match or exceed physicians. Real-world use, however, centers on treatment selection and medication guidance. The tools’ performance on those tasks remains largely unmeasured. The authors state that “evaluation and oversight frameworks should be weighted accordingly.” They call for linking AI use to clinical decisions and patient outcomes.
Rethinking Knowledge and Education
The authors observe that “sustained, widespread use suggests these tools address an unmet need in clinical practice.” Board examinations and maintenance-of-certification assessments are usually limited to a physician’s own specialty. They may therefore miss the cross-disciplinary information needs these data reveal, which has implications for continuing medical education.
More broadly, the authors conclude that “a growing share of knowledge access is shifting from recall to retrieval.” Whether this shift improves care will depend on three things: the tools’ accuracy, how clinicians are trained to use them, and how institutions choose to govern them.
Limitations
The study examined a single platform, so overall clinician use of AI is likely underestimated. Classification relied on an LLM rather than human annotation. The data could not show whether queries occurred during patient encounters. The study also could not determine whether AI was replacing traditional references or specialty consultation. Finally, the platform’s content partnerships with major publishers may limit how far the findings generalize to other tools.
Conclusion
For private practices and clinical leaders, the message is that AI-enabled clinical information tools are already part of routine care. Trainees, early-career physicians, nurse practitioners, and physician assistants are leading adoption. Governance, education, and quality oversight should reflect how these tools are actually used: mainly for treatment decisions and questions outside a clinician’s specialty.
