{"id":645918,"date":"2026-08-19T18:49:13","date_gmt":"2026-08-19T18:49:13","guid":{"rendered":"https:\/\/www.europesays.com\/ie\/645918\/"},"modified":"2026-08-19T18:49:13","modified_gmt":"2026-08-19T18:49:13","slug":"safety-and-security-of-large-language-models-in-healthcare","status":"publish","type":"post","link":"https:\/\/www.europesays.com\/ie\/645918\/","title":{"rendered":"Safety and security of large language models in healthcare"},"content":{"rendered":"
  • \n

    Gommers, J. et al. Interval cancer, sensitivity, and specificity comparing AI-supported mammography screening with standard double reading without AI in the MASAI study: a randomised, controlled, non-inferiority, single-blinded, population-based, screening-accuracy trial. Lancet 407<\/b>, 505\u2013514 (2026).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Lu, M. Y. et al. A multimodal generative AI copilot for human pathology. Nature 643<\/b>, 466\u2013473 (2024).<\/p>\n

    Article<\/a>\u00a0
    \n     ADS<\/a>\u00a0
    \n     CAS<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Singh, R., Bapna, M., Diab, A. R., Ruiz, E. S. & Lotter, W. How AI is used in FDA-authorized medical devices: a taxonomy across 1,016 authorizations. NPJ Digit. Med. 8<\/b>, 388 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Griot, M., Vanderdonckt, J. & Yuksel, D. Implementation of large language models in electronic health records. PLOS Digital Health https:\/\/doi.org\/10.1371\/journal.pdig.0001141<\/a> (2025).<\/p>\n<\/li>\n

  • \n

    Armitage, H. Clinicians can \u2018chat\u2019 with medical records through new AI software, ChatEHR. Stanford Medicine https:\/\/med.stanford.edu\/news\/all-news\/2025\/06\/chatehr.html<\/a> (2025).<\/p>\n<\/li>\n

  • \n

    Moor, M. et al. Foundation models for generalist medical artificial intelligence. Nature 616<\/b>, 259\u2013265 (2023).<\/p>\n

    Article<\/a>\u00a0
    \n     ADS<\/a>\u00a0
    \n     CAS<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Clusmann, J. et al. The future landscape of large language models in medicine. Commun. Med. 3<\/b>, 141 (2023).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Bubeck, S. et al. Sparks of Artificial General Intelligence: Early experiments with GPT-4. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2303.12712<\/a> (2023).<\/p>\n<\/li>\n

  • \n

    Singhal, K. et al. Toward expert-level medical question answering with large language models. Nat. Med. 31<\/b>, 943\u2013950 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     CAS<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Gu, Y. et al. The illusion of readiness: stress testing large frontier models on multimodal medical benchmarks. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2509.18234<\/a> (2025). This study reveals that medical LLM benchmarks may overestimate real-world readiness as benchmarks fail to capture brittleness and reasoning flaws.<\/b><\/p>\n<\/li>\n

  • \n

    Omar, M. et al. Multi-model assurance analysis showing large language models are highly vulnerable to adversarial hallucination attacks during clinical decision support. Commun. Med. 5<\/b>, 330 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Bedi, S., Jiang, Y., Chung, P., Koyejo, S. & Shah, N. Fidelity of medical reasoning in large language models. JAMA Netw. Open 8<\/b>, e2526021 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Goh, E. et al. GPT-4 assistance for improvement of physician performance on patient care tasks: a randomized controlled trial. Nat. Med. 31<\/b>, 1233\u20131238 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     CAS<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    McDuff, D. Towards accurate differential diagnosis with large language models. Nature 642<\/b>, 451\u2013457 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     ADS<\/a>\u00a0
    \n     CAS<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Tu, T. et al. Towards conversational diagnostic artificial intelligence. Nature 642<\/b>, 442\u2013450 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     ADS<\/a>\u00a0
    \n     CAS<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Palepu, A., et al. Exploring large language models for specialist-level oncology care. NEJM AI 2<\/b>, AIcs2500025 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Singhal, K. et al. Large language models encode clinical knowledge. Nature 620<\/b>, 172\u2013180 (2023).<\/p>\n

    Article<\/a>\u00a0
    \n     ADS<\/a>\u00a0
    \n     CAS<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Thirunavukarasu, A. J. et al. Large language models in medicine. Nat. Med. 29<\/b>, 1930\u20131940 (2023).<\/p>\n

    Article<\/a>\u00a0
    \n     CAS<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Van Veen, D. et al. Adapted large language models can outperform medical experts in clinical text summarization. Nat. Med. 30<\/b>, 1134\u20131142 (2024).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Tierney, A. A. et al. Ambient artificial intelligence scribes: learnings after 1 year and over 2.5 million uses. NEJM Catal. Innov. Care Deliv. https:\/\/doi.org\/10.1056\/CAT.25.0040<\/a> (2025).<\/p>\n

    Article<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Heinz, M. V. et al. Randomized trial of a generative AI chatbot for mental health treatment. NEJM AI 2<\/b>, AIoa2400802 (2025).<\/p>\n


    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Comanici, G. et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2507.06261<\/a> (2025).<\/p>\n<\/li>\n

  • \n

    Zakka, C. et al. Almanac \u2014 Retrieval-Augmented Language Models for Clinical Medicine. NEJM AI 1<\/b>, AIoa2300068 (2024).<\/p>\n

    Article<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Wiest, I. C. et al. Large language models for clinical decision support in gastroenterology and hepatology. Nat. Rev. Gastroeneterol. Hepatol. 22<\/b>, 773\u2013787 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Ferber, D. et al. Development and validation of an autonomous artificial intelligence agent for clinical decision-making in oncology. Nat. Cancer 6<\/b>, 1337\u20131349 (2025). This study presents one of the first modular LLM-agent frameworks for healthcare, integrating biomedical tools and knowledge bases for clinical dec<\/b>ision-making.<\/b><\/p>\n

    Article<\/a>\u00a0
    \n     CAS<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Ge, J. et al. Development of a liver disease-specific large language model chat interface using retrieval-augmented generation. Hepatology 80<\/b>, 1158\u20131168 (2024).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Kresevic, S. et al. Optimization of hepatological clinical guidelines interpretation by large language models: a retrieval augmented generation-based framework. NPJ Digit. Med. 7<\/b>, 102 (2024).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Masanneck, L., Meuth, S. G. & Pawlitzki, M. Evaluating base and retrieval augmented LLMs with document or online support for evidence based neurology. NPJ Digit. Med. 8<\/b>, 137 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Zakka, C. et al. Almanac Copilot: towards autonomous electronic health record navigation. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2405.07896<\/a> (2024).<\/p>\n<\/li>\n

  • \n

    Guo, D. et al. DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning. Nature 645<\/b>, 633\u2013638 (2025). This study shows that reinforcement learning can markedly improve LLM reasoning performance and introduces DeepSeek-R1 as a major open-weight model and one of few LLMs in peer-reviewed literat<\/b>ure.<\/b><\/p>\n

    Article<\/a>\u00a0
    \n     ADS<\/a>\u00a0
    \n     CAS<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Angus, D. C. et al. AI, health, and health care today and tomorrow: the JAMA Summit report on artificial intelligence. JAMA 334<\/b>, 1650\u20131664 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Garc\u00eda-Garc\u00eda, D., Le\u00f3n-G\u00f3mez, I., P\u00e9rez-Mar\u00edn, L. & G\u00f3mez-Barroso, D. Exploring all-cause mortality surveillance during the Iberian Peninsula power outage, Spain, 28 April 2025. Eurosurveillance 30<\/b>, 2500405 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Larsen, E., Fong, A., Wernz, C. & Ratwani, R. M. Implications of electronic health record downtime: an analysis of patient safety event reports. J. Am. Med. Inform. Assoc. 25<\/b>, 187\u2013191 (2018).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Martin, G., Ghafur, S., Kinross, J., Hankin, C. & Darzi, A. WannaCry\u2014a year on. BMJ 361<\/b>, k2381 (2018).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Law, R. Cyberattacks on healthcare: Russia\u2019s tool for mass disruption. Medical Device Network https:\/\/www.medicaldevice-network.com\/features\/cyberattacks-on-healthcare-russias-tool-for-mass-disruption\/<\/a> (2024).<\/p>\n<\/li>\n

  • \n

    Cartwright, A. J. The elephant in the room: cybersecurity in healthcare. J. Clin. Monit. Comput. 37<\/b>, 1123\u20131132 (2023).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Gordon, W. J. et al. Assessment of employee susceptibility to phishing attacks at US health care institutions. JAMA Netw. Open 2<\/b>, e190393 (2019).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Li, S., Surineni, K. & Prabhakaran, N. Cyber-attacks on hospital systems: a narrative review. Am. J. Ger. Psychiatry 7<\/b>, 30\u201339 (2025).<\/p>\n


    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Bowsher, G., Sullivan, R. & Lentzos, F. Tackling health disinformation in conflict settings. Lancet 405<\/b>, 1052 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Menz, B. D., Modi, N. D., Sorich, M. J. & Hopkins, A. M. Health disinformation use case highlighting the urgent need for artificial intelligence vigilance: weapons of mass disinformation: weapons of mass disinformation. JAMA Intern. Med. 184<\/b>, 92\u201396 (2024).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Perakslis, E. D., Ranney, M. L. & Goldsack, J. C. Characterizing cyber harms from digital health. Nat. Med. 29<\/b>, 528\u2013531 (2023).<\/p>\n

    Article<\/a>\u00a0
    \n     CAS<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Nweke, L. O. Using the CIA and AAA Models to Explain Cybersecurity Activities. Preprint at https:\/\/pmworldlibrary.net\/wp-content\/uploads\/2017\/05\/171126-Nweke-Using-CIA-and-AAA-Models-to-explain-Cybersecurity.pdf<\/a> (2017).<\/p>\n<\/li>\n

  • \n

    Nasr, M. et al. Scalable Extraction of Training Data from Algined, Production Language Models. ICLR 2025 https:\/\/openreview.net\/forum?id=vjel3nWP2a<\/a> (2025).<\/p>\n<\/li>\n

  • \n

    Jha, R., Zhang, C., Shmatikov, V. & Morris, J. X. Harnessing the universal geometry of embeddings. NeurIPS 2025 Conference https:\/\/neurips.cc\/virtual\/2025\/loc\/san-diego\/poster\/116441<\/a> (NeurIPS, 2025).<\/p>\n<\/li>\n

  • \n

    Finlayson, S. G., Chung, H. W., Kohane, I. S. & Beam, A. L. Adversarial attacks against medical deep learning systems. Preprint at https:\/\/doi.org\/10.48550\/arXiv.1804.05296<\/a> (2018). This early and influential work identifies adversarial attacks as a fundamental risk for medical machine learning in healthcare.<\/b><\/p>\n<\/li>\n

  • \n

    Chakraborty, A., Alam, M., Dey, V., Chattopadhyay, A. & Mukhopadhyay, D. Adversarial attacks and defences: a survey. Preprint at https:\/\/doi.org\/10.48550\/arXiv.1810.00069<\/a> (2018).<\/p>\n<\/li>\n

  • \n

    Xu, H. et al. Adversarial attacks and defenses in images, graphs and text: a review. Int. J. Autom. Comput. 17<\/b>, 151\u2013178 (2020).<\/p>\n

    Article<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Barreno, M., Nelson, B., Joseph, A. D. & Tygar, J. D. The security of machine learning. Mach. Learn. 81<\/b>, 121\u2013148 (2010).<\/p>\n

    Article<\/a>\u00a0
    \n     ADS<\/a>\u00a0
    \n     MathSciNet<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    OWASP Top 10 for LLM Applications 2025 https:\/\/genai.owasp.org\/resource\/owasp-top-10-for-llm-applications-2025\/<\/a> (OWASP, 2024). This resource provides a continuously updated overview of the most critical safety and security risks in LLMs.<\/b><\/p>\n<\/li>\n

  • \n

    CVSS v4.0 Specification Document. FIRST https:\/\/www.first.org\/cvss\/specification-document<\/a> (2023).<\/p>\n<\/li>\n

  • \n

    Kaviani, S., Han, K. J. & Sohn, I. Adversarial attacks and defenses on AI in medical imaging informatics: A survey. Expert Syst. Appl. 198<\/b>, 116815 (2022).<\/p>\n

    Article<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Nagaraja, N. & Bahsi, H. Cyber threat modeling of an LLM-based healthcare system. In Proc. 11th International Conference on Information Systems Security and Privacy 325\u2013336 (SCITEPRESS, 2025).<\/p>\n<\/li>\n

  • \n

    Li, M. Q. & Fung, B. C. M. Security concerns for large language models: a survey. Journal of Information Security and Applications, Volume 95, 2025 https:\/\/doi.org\/10.1016\/j.jisa.2025.104284<\/a> (2025).<\/p>\n<\/li>\n

  • \n

    Reason, J. Human error: models and management. Brit. Med. J. 320<\/b>, 768\u2013770 (2000).<\/p>\n

    Article<\/a>\u00a0
    \n     CAS<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Joint Task Force Transformation Initiative. Guide for Conducting Risk Assessments https:\/\/doi.org\/10.6028\/nist.sp.800-30r1<\/a> (US Department of Commerce, 2012).<\/p>\n<\/li>\n

  • \n

    Ashkenazy, S. Why time-to-market is overtaking device security as a top priority. Cybellum https:\/\/cybellum.com\/blog\/why-time-to-market-is-overtaking-device-security-as-a-top-priority-in-2025\/<\/a> (2024).<\/p>\n<\/li>\n

  • \n

    Piao, Y., Li, J. & Woods, D. W. Measuring the vulnerability disclosure policies of AI vendors. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2509.06136<\/a> (2025).<\/p>\n<\/li>\n

  • \n

    Bommasani, R. et al. The 2024 Foundation Model Transparency Index. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2407.12929<\/a> (2024).<\/p>\n<\/li>\n

  • \n

    Anthropic\u2019s responsible scaling policy. Anthropic https:\/\/www.anthropic.com\/rsp-updates<\/a> (2026).<\/p>\n<\/li>\n

  • \n

    Lindsey, J. et al. On the biology of a large language model. Anthropic https:\/\/transformer-circuits.pub\/2025\/attribution-graphs\/biology.html<\/a> (2025). This work provides an extensive analysis of internal mechanisms in LLMs and introduces open-source tools to advance mechanistic interpretability.<\/b><\/p>\n<\/li>\n

  • \n

    Petri: an open-source auditing tool to accelerate AI safety research. Anthropic https:\/\/www.anthropic.com\/research\/petri-open-source-auditing<\/a> (2025).<\/p>\n<\/li>\n

  • \n

    Joglekar, M. et al. Training LLMs for honesty via confessions. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2512.08093<\/a> (2025).<\/p>\n<\/li>\n

  • \n

    Villalobos, P. et al. Position: will we run out of data? Limits of LLM scaling based on human-generated data. ICML 235<\/b>, 49523\u201349544 (2024).<\/p>\n


    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Alber, D. A. et al. Medical large language models are vulnerable to data-poisoning attacks. Nat. Med. 31<\/b>, 618\u2013626 (2025). This study demonstrates that even minimal data poisoning can induce clinically harmful behaviour in medical LLMs.<\/b><\/p>\n

    Article<\/a>\u00a0
    \n     CAS<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Hubinger, E. et al. Sleeper agents: training deceptive LLMs that persist through safety training. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2401.05566<\/a> (2024). This study characterizes difficult-to-detect and persistent<\/b> \u2018sleeper agent\u2019 backdoor behaviours in LLMs.<\/b><\/p>\n<\/li>\n

  • \n

    Souri, H., Fowl, L. H., Chellappa, R., Goldblum, M. & Goldstein, T. in Advances in Neural Information Processing Systems 35 (eds Koyejo, S. et al.) 19165\u201319178 (NeurIPS, 2022).<\/p>\n<\/li>\n

  • \n

    Carlini, N. et al. Poisoning Web-Scale Training Datasets is Practical. in 2024 IEEE Symposium on Security and Privacy Vol. 29, 407\u2013425 (IEEE, 2024).<\/p>\n<\/li>\n

  • \n

    Clusmann, J. et al. Incidental prompt injections on vision\u2013language models in real-life histopathology. NEJM AI 2<\/b>, AIcs2500078 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Han, T. et al. Medical large language models are susceptible to targeted misinformation attacks. npj Digit. Med. 7<\/b>, 288 (2024). This study demonstrates the feasibility of targeted model weight manipulation to introduce incorrect biomedical information into LLMs.<\/b><\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Rao, P. S. B., \u0160\u0107epanovi\u0107, S., Jayagopi, D. B., Cherubini, M. & Quercia, D. The AI model risk catalog: what developers and researchers miss about real-world AI harms. In Proc. AAAI\/ACM Conference on AI, Ethics, and Society Vol. 8, 2163\u20132150 (AAAI\/ACM, 2025).<\/p>\n<\/li>\n

  • \n

    Carlini, N. et al. Poisoning web-scale training datasets is practical. ar5iv https:\/\/ar5iv.labs.arxiv.org\/html\/2302.10149<\/a> (2024).<\/p>\n<\/li>\n

  • \n

    Clusmann, J. et al. Prompt injection attacks on vision language models in oncology. Nat. Commun. 16<\/b>, 1239 (2025). This study demonstrates prompt injection attacks on LLMs through hidden prompt instructions on medical imaging data.<\/b><\/p>\n

    Article<\/a>\u00a0
    \n     ADS<\/a>\u00a0
    \n     CAS<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Zhang, Z., Qadir, M.I., Carstens, M. et al. Prompt injection attacks on vision-language models for surgical decision support. npj Digit. Surg. 1<\/b>, 15 https:\/\/doi.org\/10.1038\/s44484-026-00014-6<\/a> (2026).<\/p>\n<\/li>\n

  • \n

    Lapuschkin, S. et al. Unmasking Clever Hans predictors and assessing what machines really learn. Nat. Commun. 10<\/b>, 1096 (2019).<\/p>\n

    Article<\/a>\u00a0
    \n     ADS<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Hou, G. et al. Evaluating robustness of large audio language models to audio injection: an empirical study. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2505.19598<\/a> (2025).<\/p>\n<\/li>\n

  • \n

    Rajeev, M. et al. Cats confuse reasoning LLM: query agnostic adversarial triggers for reasoning models. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2503.01781<\/a> (2025).<\/p>\n<\/li>\n

  • \n

    Alizadeh, M., Samei, Z., Stetsenko, D. & Gilardi, F. Simple prompt injection attacks can leak personal data observed by LLM agents during task execution. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2506.01055<\/a> (2025).<\/p>\n<\/li>\n

  • \n

    Meincke, L. et al. Persuading large language models to comply with objectionable requests. Proc. Natl Acad. Sci. USA 123<\/b>, e2535868123 (2025).<\/p>\n<\/li>\n

  • \n

    Zeng, Y. et al. How Johnny can persuade LLMs to jailbreak them: rethinking persuasion to challenge AI safety by humanizing LLMs. In Proc. 62nd Meeting of the Association for Computational Linguistics Vol. 1, 14322\u201314350 (ACL, 2024).<\/p>\n<\/li>\n

  • \n

    Schoene, A. M. & Canca, C. \u2018For argument\u2019s sake, show me how to harm myself!\u2019: Jailbreaking LLMs in suicide and self-harm contexts. In 2025 IEEE International Symposium on Technology and Society https:\/\/doi.org\/10.1109\/ISTAS65609.2025.11269647<\/a> (IEEE, 2025).<\/p>\n<\/li>\n

  • \n

    Mehrotra, A. et al. Tree of Attacks: Jailbreaking Black-Box LLMs Automatically. In Advances in Neural Information Processing Systems 37 (eds Globerson, A. et al.) 61065\u201361105 (NeurIPS, 2024).<\/p>\n<\/li>\n

  • \n

    Jiang, F. et al. ArtPrompt: ASCII art-based jailbreak attacks against aligned LLMs. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2402.11753<\/a> (2024).<\/p>\n<\/li>\n

  • \n

    Zhang, H., Lou, Q. & Wang, Y. Towards safe AI clinicians: a comprehensive study on large language model jailbreaking in healthcare. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2501.18632<\/a> (2025).<\/p>\n<\/li>\n

  • \n

    Sharma, M. et al. Constitutional classifiers: defending against universal jailbreaks across thousands of hours of red teaming. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2501.18837<\/a> (2025).<\/p>\n<\/li>\n

  • \n

    Yuan, Y. et al. From hard refusals to safe-completions: toward output-centric safety training. Superintelligence https:\/\/doi.org\/10.70777\/si.v2i6.15625<\/a> (2025).<\/p>\n

    Article<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Zhang, Y., Juels, A., Reiter, M. K. & Ristenpart, T. Cross-VM side channels and their use to extract private keys. In Proc. 2012 ACM Conference on Computer and Communications Security https:\/\/doi.org\/10.1145\/2382196.2382230<\/a> (ACM, 2012).<\/p>\n<\/li>\n

  • \n

    Karim, H., Gupta, D. & Sitharaman, S. Securing LLM workloads with NIST AI RMF in the internet of robotic things. IEEE Access 13<\/b>, 69631\u201369649 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Huang, H., Meng, T. & Jia, W. Joint optimization of prompt security and system performance in Edge-Cloud LLM systems. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2501.18663<\/a> (2025).<\/p>\n<\/li>\n

  • \n

    Schmotz, D., Abdelnabi, S. & Andriushchenko, M. Agent skills enable a new class of realistic and trivially simple prompt injections. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2510.26328<\/a> (2025).<\/p>\n<\/li>\n

  • \n

    Narajala, V. S. & Habler, I. Enterprise-grade security for the model context protocol (MCP): frameworks and mitigation strategies. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2504.08623<\/a> (2025).<\/p>\n<\/li>\n

  • \n

    Thoonsen, A. C. et al. Stimulating implementation of clinical practice guidelines in hospital care from a central guideline organization perspective: A systematic review. Health Policy 148<\/b>, 105135 (2024).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Clusmann, J. et al. The barriers for uptake of artificial intelligence in hepatology and how to overcome them. J. Hepatol. 83<\/b>, 1410\u20131426 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Wong, E. Y. T. et al. ESMO guidance on the use of large language models in clinical practice (ELCAP). Ann. Oncol. 36<\/b>, 1447\u20131457 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     CAS<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Sarthi, P. et al. RAPTOR: recursive abstractive processing for tree-organized retrieval. ICLR https:\/\/iclr.cc\/virtual\/2024\/poster\/19034<\/a> (2024).<\/p>\n<\/li>\n

  • \n

    Greshake, K. et al. Not what you\u2019ve signed up for: compromising real-world LLM-integrated applications with indirect prompt injection. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2302.12173<\/a> (2023).<\/p>\n<\/li>\n

  • \n

    Continella, A., Polino, M., Pogliani, M. & Zanero, S. There\u2019s a hole in that bucket!: a large-scale analysis of misconfigured S3 buckets. In Proc. 34th Annual Computer Security Applications Conference 702\u2013711 (ACM, 2018).<\/p>\n<\/li>\n

  • \n

    Cruz, F. & Lombrozo, T. How laypeople evaluate scientific explanations containing jargon. Nat. Hum. Behav. 9<\/b>, 2038\u20132053 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Measuring the Persuasiveness of Language Models. Anthropic https:\/\/www.anthropic.com\/news\/measuring-model-persuasiveness<\/a> (2024).<\/p>\n<\/li>\n

  • \n

    Salvi, F., Horta Ribeiro, M., Gallotti, R. & West, R. On the conversational persuasiveness of GPT-4. Nat. Hum. Behav. 9<\/b>, 1645\u20131653 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Shekar, S., Pataranutaporn, P., Sarabu, C., Cecchi, G. A. & Maes, P. People overtrust AI-generated medical advice despite low accuracy. NEJM AI 2<\/b>, AIoa2300015 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Li, W. et al. Can a large language model be a gaslighter? In The 13th International Conference on Learning Representations (eds Yue, Y. et al.) https:\/\/proceedings.iclr.cc\/paper_files\/paper\/2025\/file\/0769598fdeb4f23ee86fec1bc0777f44-Paper-Conference.pdf<\/a> (ICLR, 2024).<\/p>\n<\/li>\n

  • \n

    Yeung, J. A., Dalmasso, J., Foschini, L., Dobson, R. J. B. & Kraljevic, Z. The psychogenic machine: simulating AI psychosis, delusion reinforcement and harm enablement in large language models. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2509.109702025<\/a> (2025).<\/p>\n<\/li>\n

  • \n

    Turpin, M., Michael, J., Perez, E. & Bowman, S. R. Language models don\u2019t always say what they think: unfaithful explanations in chain-of-thought prompting. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2305.04388<\/a> (2023).<\/p>\n<\/li>\n

  • \n

    Baker, B. et al. Monitoring reasoning models for misbehavior and the risks of promoting obfuscation. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2503.11926<\/a> (2025).<\/p>\n<\/li>\n

  • \n

    Perez, E. et al. Discovering language model behaviors with model-written evaluations. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2212.09251<\/a> (2022).<\/p>\n<\/li>\n

  • \n

    Xiao, J. et al. On the algorithmic bias of aligning large language models with RLHF: Preference collapse and matching regularization. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2405.16455<\/a> (2024).<\/p>\n<\/li>\n

  • \n

    Chen, S. et al. When helpfulness backfires: LLMs and the risk of false medical information due to sycophantic behavior. NPJ Digit. Med. 8<\/b>, 605 (2025). This study identifies sycophantic behaviour in LLMs as a potential source of false medical information in healthcare settings.<\/b><\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Kalai, A. T., Nachum, O., Vempala, S. S. & Zhang, E. Why language models hallucinate. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2509.04664<\/a> (2025). This study characterizes hallucinations in LLMs as a consequence of misaligned training objectives that reward plausible guessing over admitting uncertainty.<\/b><\/p>\n<\/li>\n

  • \n

    Sharma, M. et al. Towards understanding sycophancy in language models. In The 12th International Conference on Learning Representations (eds Kim, B. et al.) https:\/\/proceedings.iclr.cc\/paper_files\/paper\/2024\/file\/0105f7972202c1d4fb817da9f21a9663-Paper-Conference.pdf<\/a> (ICLR, 2023).<\/p>\n<\/li>\n

  • \n

    Cheng, M. et al. Sycophantic AI decreases prosocial intentions and promotes dependence. Science 391<\/b>, eaec8352 (2026).<\/p>\n

    Article<\/a>\u00a0
    \n     CAS<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Miton, H., Claidi\u00e8re, N. & Mercier, H. Universal cognitive mechanisms explain the cultural success of bloodletting. Evol. Hum. Behav. 36<\/b>, 303\u2013312 (2015).<\/p>\n

    Article<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Saposnik, G., Redelmeier, D., Ruff, C. C. & Tobler, P. N. Cognitive biases associated with medical decisions: a systematic review. BMC Med. Inform. Decis. Mak. 16<\/b>, 138 (2016).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Huang, L. et al. A survey on hallucination in large language models: principles, taxonomy, challenges, and open questions. ACM Trans. Inf. Syst. 43<\/b>, 42 (2025). This study provides a systematic analysis and taxonomy of hallucinations in LLMs.<\/b><\/p>\n

    Article<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Ferber, D. et al. In-context learning enables multimodal large language models to classify cancer pathology images. Nat. Commun. 15<\/b>, 10104 (2024).<\/p>\n

    Article<\/a>\u00a0
    \n     ADS<\/a>\u00a0
    \n     CAS<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Gourabathina, A., Gerych, W., Pan, E. & Ghassemi, M. The medium is the message: how non-clinical information shapes clinical decisions in LLMs. In Proc. 2025 ACM Conference on Fairness, Accountability, and Transparency 1805\u20131828 (ACM, 2025).<\/p>\n<\/li>\n

  • \n

    Coda-Forno, J. et al. Inducing anxiety in large language models can induce bias. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2304.11111<\/a> (2023).<\/p>\n<\/li>\n

  • \n

    Corbeil, J.-P., Kim, M., Sordoni, A., Beaulieu, F. & Vozila, P. Medical red teaming protocol of language models: on the importance of user perspectives in healthcare settings. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2507.07248<\/a> (2025).<\/p>\n<\/li>\n

  • \n

    Callahan, A. et al. Standing on FURM ground: a framework for evaluating fair, useful, and reliable AI models in health care systems. NEJM Catal. Innov. Care Deliv. https:\/\/doi.org\/10.1056\/CAT.24.0131<\/a> (2024).<\/p>\n

    Article<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Huang, Y. et al. Position: TrustLLM: trustworthiness in large language models. In Proc. 41st International Conference on Machine Learning 20166\u201320270 (PMLR, 2024).<\/p>\n<\/li>\n

  • \n

    Gabriel, I. Artificial intelligence, values, and alignment. Minds Mach. 30<\/b>, 411\u2013437 (2020).<\/p>\n

    Article<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    M\u00f6kander, J., Schuett, J., Kirk, H. R. & Floridi, L. Auditing large language models: a three-layered approach. AI Ethics 4<\/b>, 1085\u20131115 (2023).<\/p>\n

    Article<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Wu, D. et al. First, do NOHARM: towards clinically safe large language models. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2512.01241<\/a> (2025).<\/p>\n<\/li>\n

  • \n

    Bedi, S. et al. Holistic evaluation of large language models for medical tasks with MedHELM. Nat. Med. 32<\/b>, 943\u2013951 (2026). This work proposes MedHELM, a comprehensive holistic evaluation suite for LLMs for medicine.<\/b><\/p>\n

    Article<\/a>\u00a0
    \n     CAS<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Han, T., Kumar, A., Agarwal, C. & Lakkaraju, H. in ICML 2024 Workshop on Models of Human Feedback for AI Alignment https:\/\/icml.cc\/virtual\/2024\/39424<\/a> (2024).<\/p>\n<\/li>\n

  • \n

    Shihab, I. F., Akter, S. & Sharma, A. Detecting and mitigating reward hacking in Reinforcement Learning systems: A comprehensive empirical study. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2507.05619<\/a> (2025).<\/p>\n<\/li>\n

  • \n

    Wu, S. et al. A comparative study on reasoning patterns of OpenAI\u2019s o1 model. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2410.13639<\/a> (2024).<\/p>\n<\/li>\n

  • \n

    El, B. & Zou, J. Moloch\u2019s bargain: emergent misalignment when LLMs compete for audiences. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2510.06105<\/a> (2025).<\/p>\n<\/li>\n

  • \n

    Betley, J. et al. Training large language models on narrow tasks can lead to broad misalignment. Nature 649<\/b>, 584\u2013589 (2026). This study characterizes<\/b> \u2018emergent misalignment\u2019 and shows that that fine-tuning LLMs on narrow tasks can induce broad, unintended and harmful behavio<\/b>urs.<\/b><\/p>\n

    Article<\/a>\u00a0
    \n     ADS<\/a>\u00a0
    \n     CAS<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Truhn, D., Reis-Filho, J. S. & Kather, J. N. Large language models should be used as scientific reasoning engines, not knowledge databases. Nat. Med. 29<\/b>, 2983\u20132984 (2023).<\/p>\n

    Article<\/a>\u00a0
    \n     CAS<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Asgari, E. et al. A framework to assess clinical safety and hallucination rates of LLMs for medical text summarisation. NPJ Digit. Med. 8<\/b>, 274 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Kim, Y. et al. Medical hallucinations in foundation models and their impact on healthcare. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2503.05777<\/a> (2025).<\/p>\n<\/li>\n

  • \n

    Bang, Y. et al. HalluLens: LLM hallucination benchmark. In Proc. 63rd Meeting of the Association for Computational Linguistics Vol. 1, 24128\u201324156 (ACL, 2025).<\/p>\n<\/li>\n

  • \n

    Soffer, S., Sorin, V., Nadkarni, G. N. & Klang, E. Pitfalls of large language models in medical ethics reasoning. NPJ Digit. Med. 8<\/b>, 461 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Xu, H. et al. Reducing tool hallucination via reliability alignment. In Proc. 42nd International Conference on Machine Learning 2799 (ACM, 2025).<\/p>\n<\/li>\n

  • \n

    Chung, P. et al. Verifying facts in patient care documents generated by large language models using electronic health records. NEJM AI 3<\/b>, AIdbp2500418 (2025).<\/p>\n


    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    World Medical Association. WMA Declaration of Helsinki\u2014Ethical Principles for Medical Research Involving Human Participants. WMA https:\/\/www.wma.net\/policies-post\/wma-declaration-of-helsinki<\/a> (2024).<\/p>\n<\/li>\n

  • \n

    Yu, K.-H., Healey, E., Leong, T.-Y., Kohane, I. S. & Manrai, A. K. Medical artificial intelligence and human values. N. Engl. J. Med. 390<\/b>, 1895\u20131904 (2024).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Mazeika, M. et al. Utility engineering: Analyzing and controlling emergent value systems in AIs. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2502.08640<\/a> (2025).<\/p>\n<\/li>\n

  • \n

    Greenblatt, R. et al. Alignment faking in large language models. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2412.14093<\/a> (2024).<\/p>\n<\/li>\n

  • \n

    Bedi, S. et al. Testing and evaluation of health care applications of large language models: A systematic review: A systematic review. JAMA 333, 319\u2013328 (2024).<\/p>\n

    Article<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Zack, T. et al. Assessing the potential of GPT-4 to perpetuate racial and gender biases in health care: a model evaluation study. Lancet Digit. Health 6<\/b>, e12\u2013e22 (2024). This study highlights that LLMs can perpetuate racial and gender biases beyond evidence-based variation.<\/b><\/p>\n

    Article<\/a>\u00a0
    \n     CAS<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Omar, M. et al. Sociodemographic biases in medical decision making by large language models. Nat. Med. 31<\/b>, 1873\u20131881 (2025). This study assesses sociodemographic biases in medical LLMs, demonstrating differences in clinical decision-making that extend beyond evidence-based variation.<\/b><\/p>\n

    Article<\/a>\u00a0
    \n     CAS<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Gruber, V.-E. et al. A women\u2019s health benchmark for large language models. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2512.17028<\/a> (2025).<\/p>\n<\/li>\n

  • \n

    Yang, J., Soltan, A. A. S., Eyre, D. W., Yang, Y. & Clifton, D. A. An adversarial training framework for mitigating algorithmic biases in clinical machine learning. NPJ Digit. Med. 6<\/b>, 55 (2023).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Gichoya, J. W. et al. AI pitfalls and what not to do: mitigating bias in AI. Br. J. Radiol. 96<\/b>, 20230023 (2023).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Ktena, I. et al. Generative models improve fairness of medical classifiers under distribution shifts. Nat. Med. 30<\/b>, 1166\u20131173 (2024).<\/p>\n

    Article<\/a>\u00a0
    \n     CAS<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Xu, Z. Mitigating social bias in large language models: a multi-objective approach within a multi-agent framework. In Proc. 39th Conf. AAAI Artificial Intelligence 25587\u201325579 (AAAI, 2025).<\/p>\n<\/li>\n

  • \n

    Binz, M. et al. A foundation model to predict and capture human cognition. Nature 644<\/b>, 1002\u20131009 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     ADS<\/a>\u00a0
    \n     CAS<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Mendu, S. K., Yenala, H., Gulati, A., Kumar, S. & Agrawal, P. Towards safer pretraining: analyzing and filtering harmful content in webscale datasets for responsible LLMs. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2505.02009<\/a> (2025).<\/p>\n<\/li>\n

  • \n

    Griot, M., Hemptinne, C., Vanderdonckt, J. & Yuksel, D. Large language models lack essential metacognition for reliable medical reasoning. Nat. Commun. 16<\/b>, 642 (2025). This study reveals a gap between benchmark performance and metacognitive awareness in LLMs.<\/b><\/p>\n

    Article<\/a>\u00a0
    \n     ADS<\/a>\u00a0
    \n     CAS<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    van Buchem, M. M. et al. Impact of a digital scribe system on clinical documentation time and quality: Usability study. JMIR AI 3<\/b>, e60020 (2024).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Ma, S. P. et al. Ambient artificial intelligence scribes: utilization and impact on documentation time. J. Am. Med. Inform. Assoc. 32<\/b>, 381\u2013385 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    You, J. G. et al. Ambient documentation technology in clinician experience of documentation burden and burnout. JAMA Netw. Open 8<\/b>, e2528056 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Blease, C. R., Locher, C., Gaab, J., H\u00e4gglund, M. & Mandl, K. D. Generative artificial intelligence in primary care: an online survey of UK general practitioners. BMJ Health Care Inform. 31<\/b>, e101102 (2024).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Eppler, M. et al. Awareness and use of ChatGPT and large language models: a prospective cross-sectional global survey in urology. Eur. Urol. 85<\/b>, 146\u2013153 (2024).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Poon, E. G., Lemak, C. H., Rojas, J. C., Guptill, J. & Classen, D. Adoption of artificial intelligence in healthcare: survey of health system priorities, successes, and challenges. J. Am. Med. Inform. Assoc. 32<\/b>, 1093\u20131100 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Egli, S. B., Arpagaus, A., Amacher, S. A., Hunziker, S. & Bassetti, S. Use, knowledge and perception of large language models in clinical practice: a cross-sectional mixed-methods survey among clinicians in Switzerland. BMJ Health Care Inform. 32<\/b>, e101470 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Castiblanco Jimenez, I. A., Gomez Acevedo, J. S., Marcolin, F., Vezzetti, E. & Moos, S. Towards an integrated framework to measure user engagement with interactive or physical products. Int. J. Interact. Des. Manuf. 17<\/b>, 45\u201367 (2023).<\/p>\n

    Article<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Chang, C. T. et al. Red teaming ChatGPT in medicine to yield real-world insights on model behavior. NPJ Digit. Med. 8<\/b>, 149 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Artsi, Y. et al. Large language models in real-world clinical workflows: a systematic review of applications and implementation. Front. Digit. Health 7<\/b>, 1659134 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Bondi-Kelly, E. et al. Taking off with AI: lessons from aviation for healthcare. In Proc. 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization https:\/\/doi.org\/10.1145\/3617694.3623224<\/a> (ACM, 2023).<\/p>\n<\/li>\n

  • \n

    Kolbinger, F. R. & Kather, J. N. Adaptive validation strategies for real-world clinical artificial intelligence. Nat. Comput. Sci. 5<\/b>, 980\u2013986 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Crowe, B. et al. Recommendations for clinicians, technologists, and healthcare organizations on the use of generative artificial intelligence in medicine: a position statement from the Society of General Internal Medicine. J. Gen. Intern. Med. 40<\/b>, 694\u2013702 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Zhou, A. et al. AutoRedTeamer: autonomous red teaming with lifelong attack integration. NeurIPS 2025 Conference https:\/\/openreview.net\/forum?id=xQH4lDLIC0<\/a> (NeurIPS, 2025).<\/p>\n<\/li>\n

  • \n

    Bastani, H. et al. Generative AI without guardrails can harm learning: evidence from high school mathematics. Proc. Natl Acad. Sci. USA 122<\/b>, e2422633122 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     CAS<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Mollick, E. R. et al. AI Agents and Education: Simulated Practice at Scale. The Wharton School Research Paper https:\/\/doi.org\/10.2139\/ssrn.4871171<\/a> (SSRN, 2024).<\/p>\n<\/li>\n

  • \n

    Kosmyna, N. et al. Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant for essay writing task. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2506.08872<\/a> (2025).<\/p>\n<\/li>\n

  • \n

    Hoffmann, M., Boysel, S., Nagle, F., Peng, S. & Xu, K. Generative AI and the nature of work. Harvard Business School Working Paper 25-021 https:\/\/doi.org\/10.2139\/ssrn.5007084<\/a> (SSRN, 2024).<\/p>\n<\/li>\n

  • \n

    Wekenborg, M. K., Gilbert, S. & Kather, J. N. Examining human\u2013AI interaction in real-world healthcare beyond the laboratory. NPJ Digit. Med. 8<\/b>, 169 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Berzin, T. M. & Topol, E. J. Preserving clinical skills in the age of AI assistance. Lancet 406<\/b>, 1719 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Budzy\u0144, K. et al. Endoscopist deskilling risk after exposure to artificial intelligence in colonoscopy: a multicentre, observational study. Lancet Gastroenterol. Hepatol. 10<\/b>, 896\u2013903 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Huo, W., Li, Q., Liang, B., Wang, Y. & Li, X. When healthcare professionals use AI: exploring work well-being through psychological needs satisfaction and job complexity. Behav. Sci. 15<\/b>, 88 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Rafailov, R. et al. Direct preference optimization: your language model is secretly a reward model. NeurIPS 2023 Conference https:\/\/neurips.cc\/virtual\/2023\/poster\/72164<\/a> (NeurIPS, 2023).<\/p>\n<\/li>\n

  • \n

    Zou, A. et al. Representation engineering: a top-down approach to AI transparency https:\/\/doi.org\/10.48550\/arXiv.2310.01405<\/a> (2023).<\/p>\n<\/li>\n

  • \n

    Zou, A. et al. Improving alignment and robustness with circuit breakers. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2406.04313<\/a> (2024).<\/p>\n<\/li>\n

  • \n

    Inan, H. et al. Llama Guard: LLM-based input\u2013output safeguard for Human\u2013AI conversations. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2312.06674<\/a> (2023).<\/p>\n<\/li>\n

  • \n

    Rebedea, T., Dinu, R., Sreedhar, M. N., Parisien, C. & Cohen, J. NeMo guardrails: a toolkit for controllable and safe LLM applications with programmable rails. In Proc. 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations 431\u2013445 (Association for Computational Linguistics, 2023).<\/p>\n<\/li>\n

  • \n

    Alaa, A. et al. Position: medical large language model benchmarks should prioritize construct validity Oral. In Proc. International Conference on Machine Learning 2025 https:\/\/icml.cc\/virtual\/2025\/oral\/40130<\/a> (ICML, 2025).<\/p>\n<\/li>\n

  • \n

    Gallifant, J. & Bitterman, D. S. Humanity\u2019s next medical exam: preparing to evaluate superhuman systems. NEJM AI 2<\/b>, AIe2501008 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Freyer, O. et al. Consideration of cybersecurity risks in the benefit-risk analysis of medical devices: scoping review. J. Med. Internet Res. 26<\/b>, e65528 (2024).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Ostermann, M. et al. Cybersecurity requirements for medical devices in the EU and US\u2014a comparison and gap analysis of the MDCG 2019-16 and FDA premarket cybersecurity guidance. Comput. Struct. Biotechnol. J. 28<\/b>, 259\u2013266 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Moberly, T. Doctors must stop using unregistered AI scribe tools, says NHS England. Brit. Med. J. 389<\/b>, r1302 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Freyer, O., Wiest, I. C., Kather, J. N. & Gilbert, S. A future role for health applications of large language models depends on regulators enforcing safety standards. Lancet Digit. Health 6<\/b>, e662\u2013e672 (2024).<\/p>\n

    Article<\/a>\u00a0
    \n     CAS<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Biasin, E., Kamenja\u0161evi\u0107, E. & Ludvigsen, K. R. in Research Handbook on Health, AI and the Law (eds Solaiman, B. & Cohen, I. G.) 57\u201374 (Edward Elgar, 2024).<\/p>\n<\/li>\n

  • \n

    Freyer, O., Jayabalan, S., Kather, J. N. & Gilbert, S. Overcoming regulatory barriers to the implementation of AI agents in healthcare. Nat. Med. 31<\/b>, 3239\u20133243 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     CAS<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Mathias, R., Schonfelder, A., Welzel, C. & Gilbert, S. Letter to the editor on \u2018From concept to clinic: living labs and regulatory sandboxes for health system digitalization and the integration of innovative devices into clinical workflows\u2019. IEEE J. Transl. Eng. Health Med. 13<\/b>, 214\u2013215 (2025).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Adler-Milstein, J. et al. Electronic health record adoption in US hospitals: the emergence of a digital \u2018advanced use\u2019 divide. J. Am. Med. Inform. Assoc. 24<\/b>, 1142\u20131148 (2017).<\/p>\n

    Article<\/a>\u00a0
    \n     PubMed<\/a>\u00a0
    \n     PubMed Central<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Hwang, Y.-M., Ng, M. Y., Pillai, M., Sahai, M. P. & Hernandez-Boussard, T. The landscape of AI implementation in US hospitals. Nat. Health 1<\/b>, 99\u2013112 (2026).<\/p>\n

    Article<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Shanahan, M. Talking about large language models. Commun. ACM 67<\/b>, 68\u201379 (2024).<\/p>\n

    Article<\/a>\u00a0
    \n
    \n Google Scholar<\/a>\u00a0\n <\/p>\n<\/li>\n

  • \n

    Ferber, D. et al. Towards autonomous medical artificial intelligence agents. Nature 655<\/b>, 1282\u20131291 (2026).<\/p>\n<\/li>\n","protected":false},"excerpt":{"rendered":"Gommers, J. et al. Interval cancer, sensitivity, and specificity comparing AI-supported mammography screening with standard double reading without…\n","protected":false},"author":2,"featured_media":645919,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","_share_on_mastodon":"0"},"categories":[275],"tags":[4155,1096,5960,18,135,475,474,1099,19,17,183713,1100,133,3473],"class_list":["post-645918","post","type-post","status-publish","format-standard","has-post-thumbnail","category-healthcare","tag-computational-science","tag-computer-science","tag-diagnosis","tag-eire","tag-health","tag-health-care","tag-healthcare","tag-humanities-and-social-sciences","tag-ie","tag-ireland","tag-medical-ethics","tag-multidisciplinary","tag-science","tag-translational-research"],"share_on_mastodon":{"url":"https:\/\/pubeurope.com\/@ie\/117123676445751629","error":""},"_links":{"self":[{"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/posts\/645918","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/comments?post=645918"}],"version-history":[{"count":0,"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/posts\/645918\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/media\/645919"}],"wp:attachment":[{"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/media?parent=645918"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/categories?post=645918"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/tags?post=645918"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}