top of page
Image: N509FZ / Wikimedia Commons (CC BY-SA 4.0). Peking Union Medical College Hospital in Beijing, where the thoracic surgery team in the study works

MedGPT Beats Doctors in a Study Its Maker Scored With AI

Writer: tech360.tv
tech360.tv
1 hour ago
2 min read

Future Doctor says its MedGPT model produced better treatment plans than unassisted doctors in 660 of 750 comparisons, or 88 per cent, across 40 complex lung cancer cases. The Chinese company published the results itself, and Chinese media reported them last week.


Image: N509FZ / Wikimedia Commons (CC BY-SA 4.0). Peking Union Medical College Hospital in Beijing, where the thoracic surgery team in the study works
Image: N509FZ / Wikimedia Commons (CC BY-SA 4.0). Peking Union Medical College Hospital in Beijing, where the thoracic surgery team in the study works

And the design of the test is where the argument sits. The 250 doctors came from 98 centres across 27 provincial regions, and 159 of them are chief or associate chief physicians. Each handled three cases under three conditions, working alone, with a literature search, and with MedGPT, generating 2,250 treatment strategies.


And twenty multidisciplinary experts wrote 240 scoring rules across three rounds, including 40 safety limits on who could be treated and how aggressively, with any breach recorded as an automatic loss. Once those rules were frozen, three AI models scored the strategies without being told which came from where.


But those judges were GPT-5.5, Claude Opus 4.7 and Gemini 3.1 Pro, which are the same families as the general models MedGPT was measured against. Two of them are the exact versions in the comparison, so a judge scoring its own family is a conflict the company has not addressed, and it has not said whether the study was peer reviewed.


The numbers it reports are stark. Doctors averaged 53.40 working alone and 66.27 with MedGPT, a 24.1 per cent relative gain, while MedGPT itself scored 96.22 against 71.38 for GPT-5.4, 61.81 for Claude Opus 4.7 and 54.62 for Gemini 3.1 Pro.


And the mechanism is retrieval plus simulation rather than a single answer. Specialist agents analyse the case, the system drafts several candidate paths including a guideline based one and one drawn from recent research, checks them against about 201,000 lung cancer evidence units from roughly 13,000 documents, then simulates each path forward and drops the ones with unresolved risks.


And the limits are as important as the result. The cases were unpublished grey zone cases chosen where guidelines give no clear answer, the scoring measured written strategies rather than patient outcomes, and every figure here comes from the company that built the model.


Future Doctor, which grew out of Medlinker, says MedGPT was the first generative AI algorithm approved in China for disease diagnosis and treatment recommendations. That regulatory first is a stronger claim than the study, because it was decided by someone other than the company.


  • MedGPT outscored unassisted doctors in 660 of 750 comparisons

  • Doctors scored 53.40 alone and 66.27 with the model

  • Three AI judges scored the plans, two from rival model families

  • All figures are self reported and not peer reviewed

  • The study measured written strategies, not patient outcomes


Source: Pandaily

 
 

Technology increasingly permeates every facet of our lives, making informed decision making an essential pursuit. We bridge this gap by combining the precision of AI with the irreplaceable discernment of human expertise. Our team produces rigorous product reviews that offer unique insights, honest critiques, and trustworthy recommendations. We also leverage AI to synthesise complex news from reliable sources into clear, actionable updates, ensuring that every story is carefully fact checked by our editorial staff before publication. Accuracy remains our priority. Should you identify any discrepancies, please contact us at editorial@tech360.tv. Your feedback is a vital part of our process in maintaining the high standards our readers deserve.

Tech360tv is Singapore's Tech News and Gadget Reviews platform. Join us for our in depth PC reviews, Smartphone reviews, Audio reviews, Camera reviews and other gadget reviews.

  • YouTube
  • Facebook
  • TikTok
  • Instagram
  • Twitter
  • LinkedIn

© 2021 tech360.tv. All rights reserved.

bottom of page