Differential Knowledge of Free and Subscribed Chatbots on Aspergillus fumigatus, a Mold of Global Importance, and Talaromyces marneffei, a Thermally Dimorphic Fungus Associated with Tropical Infections in Southeast Asia

Zi-Jie Lee; Chi-Ching Tsang; Chun-Sheng Wang; Yu Hsiao; Susanna K.P. Lau; Patrick C.Y. Woo

doi:10.53941/emicrobe.2025.100003

Abstract

Chatbots have been widely used in clinical problem-solving and research. However, all the chatbots examined were products of the USA, and there has been no study that compared the knowledge of these chatbots on specific pathogens of global vs. regional importance. In this study, we examined the knowledge of five free chatbots (ChatGPT, Perplexity, Claude, Copilot, and Gemini) and the free vs. subscribed versions of ChatGPT, Perplexity, and Claude on Talaromyces marneffei, a thermally dimorphic pathogenic fungus of regional importance in Southeast Asia, and Aspergillus fumigatus, a mold of global importance, using 200 true/false questions on T. marneffei and A. fumigatus set and cross-validated by three full/assistant professors. There was a statistically significant difference among the median scores of the five free chatbots for the eight subsets of T. marneffei and A. fumigatus questions (p = 0.006). Dunn’s test showed that the overall score of Claude 3.5 Sonnet was significantly higher than those of Perplexity (p = 0.032) and Gemini (p = 0.008). Further analysis showed that the median score of Claude 3.5 Sonnet was higher than those of Perplexity and Gemini for both the T. marneffei (p = 0.037 and p = 0.027, respectively) and A. fumigatus questions (p = 0.137 and p = 0.058, respectively). The median score obtained by Perplexity Pro was significantly higher than that of Perplexity (p = 0.038). There was no significant difference between the scores for the chatbots in the four subsets of T. marneffei and the four subsets of A. fumigatus questions. Differential performance exists for the different free/subscribed chatbots in answering the T. marneffei and A. fumigatus questions.

References

1.
Woo, P.C. Rigorous analysis of microbes and infectious diseases using an expanding range of robust in silico Technologies. eMicrobe 2025, 1, 1.
2.
Bera, K.; Braman, N.; Gupta, A.; et al. Predicting cancer outcomes with radiomics and artificial intelligence in radiology. Nat. Rev. Clin. Oncol. 2022, 19, 132–146.
3.
Goyal, M.; Knackstedt, T.; Yan, S.; et al. Artificial intelligence-based image classification methods for diagnosis of skin cancer: Challenges and opportunities. Comput. Biol. Med. 2020, 127, 104065.
4.
Försch, S.; Klauschen, F.; Hufnagl, P.; et al. Artificial intelligence in pathology. Dtsch. Ärzteblatt Int. 2021, 118, 199.
5.
Shafi, S.; Parwani, A.V. Artificial intelligence in diagnostic pathology. Diagn. Pathol. 2023, 18, 109.
6.
Tsang, C.C.; Zhao, C.; Liu, Y.; et al. Automatic identification of clinically important Aspergillus species by artificial intelligence-based image recognition: Proof-of-concept study. Emerg. Microbes Infect. 2025, 14, 2434573.
7.
Theodosiou, A.A.; Read, R.C. Artificial intelligence, machine learning and deep learning: Potential resources for the infection clinician. J. Infect. 2023, 87, 287–294.
8.
Kumar, S.; Arif, T.; Alotaibi, A.S.; et al. Advances towards automatic detection and classification of parasites microscopic images using deep convolutional neural network: Methods, models and research directions. Arch. Comput. Methods Eng. 2023, 30, 2013–2039.
9.
Wang, Z.; Zhang, L.; Zhao, M.; et al. Deep neural networks offer morphologic classification and diagnosis of bacterial vaginosis. J. Clin. Microbiol. 2021, 59, 10-1128.
10.
Song, Y.; He, L.; Zhou, F.; et al. Segmentation, splitting, and classification of overlapping bacteria in microscope images for automatic bacterial vaginosis diagnosis. IEEE J. Biomed. Health Inform. 2016, 21, 1095–1104.
11.
Van Noorden, R.; Webb, R. ChatGPT and science: The AI system was a force in 2023−−for good and bad. Nature 2023, 624, 509.
12.
Dave, T.; Athaluri, S.A.; Singh, S. ChatGPT in medicine: An overview of its applications, advantages, limitations, future prospects, and ethical considerations. Front. Artif. Intell. 2023, 6, 1169595.
13.
Garg, R.K.; Urs, V.L.; Agarwal, A.A.; et al. Exploring the role of ChatGPT in patient care (diagnosis and treatment) and medical research: A systematic review. Health Promot. Perspect. 2023, 13, 183.
14.
Yan, M.; Cerri, G.G.; Moraes, F.Y. ChatGPT and medicine: How AI language models are shaping the future and health related careers. Nat. Biotechnol. 2023, 41, 1657–1658.
15.
Mahmoud, R.; Shuster, A.; Kleinman, S.; et al. Evaluating artificial intelligence chatbots in oral and maxillofacial surgery board exams: Performance and potential. J. Oral Maxillofac. Surg. 2025, 83, 382–389.
16.
Mayo-Yáñez, M.; Lechien, J.R.; Maria-Saibene, A.; et al. Examining the performance of ChatGPT 3.5 and microsoft copilot in otolaryngology: A comparative study with otolaryngologists’ evaluation. Indian J. Otolaryngol. Head Neck Surg. 2024, 76, 3465–3469.
17.
Wang, C.S.; Hsiao, Y.; Tsou, C.H.; et al. Chatbots are just as good as professors in both factual recall and clinical scenario analysis: Emergence of a new tool in clinical microbiology and infectious disease. J. Infect. 2024, 89, 106274.
18.
Tsang, C.C.; Lau, S.K.P.; Woo, P.C.Y. Sixty years from Segretain’s description: What have we learned and should learn about the basic mycology of Talaromyces marneffei? Mycopathologia 2019, 184, 721–729.
19.
Woo, P.C.Y.; Zhen, H.; Cai, J.J.; et al. The mitochondrial genome of the thermal dimorphic fungus Penicillium marneffei is more closely related to those of molds than yeasts. FEBS Lett. 2003, 555, 469–477.
20.
Tam, E.W.; Tsang, C.C.; Lau, S.K.P.; et al. Comparative mitogenomic and phylogenetic characterization on the complete mitogenomes of Talaromyces (Penicillium) marneffei. Mitochondrial DNA Part B 2016, 1, 941–942.
21.
Yuen, K.Y.; Woo, P.C.Y.; Ip, M.S.; et al. Stage-specific manifestation of mold infections in bone marrow transplant recipients: Risk factors and clinical significance of positive concentrated smears. Clin. Infect. Dis. 1997, 25, 37–42.
22.
Chan, J.F.W.; Lau, S.K.P.; Wong, S.C.Y.; et al. A 10-year study reveals clinical and laboratory evidence for the ‘semi-invasive’ properties of chronic pulmonary aspergillosis. Emerg. Microbes Infect. 2016, 5, 1–7.
23.
Chan, J.F.W.; Chan, T.S.Y.; Gill, H.; et al. Disseminated infections with Talaromyces marneffei in non-AIDS patients given monoclonal antibodies against CD20 and kinase inhibitors. Emerg. Infect. Dis. 2015, 21, 1101.
24.
Chan, J.F.; Lau, S.K.P.; Yuen, K.Y.; et al. Talaromyces (Penicillium) marneffei infection in non-HIV-infected patients. Emerg. Microbes Infect. 2016, 5, 1–9.
25.
Narayanasamy, S.; Dat, V.Q.; Thanh, N.T.; et al. A global call for talaromycosis to be recognised as a neglected tropical disease. Lancet Glob. Health 2021, 9, e1618–e1622.
26.
Lau, S.K.; Lam, C.S.; Ngan, A.H.; et al. Matrix-assisted laser desorption ionization time-of-flight mass spectrometry for rapid identification of mold and yeast cultures of Penicillium marneffei. BMC Microbiol. 2016, 16, 1–9.
27.
Lau, S.K.P.; Tang, B.S.; Curreem, S.O.; et al. Matrix-assisted laser desorption ionization–time of flight mass spectrometry for rapid identification of Burkholderia pseudomallei: Importance of expanding databases with pathogens endemic to different localities. J. Clin. Microbiol. 2012, 50, 3142–3143.
28.
Tang, B.S.; Lau, S.K.P.; Teng, J.L.; et al. Matrix-assisted laser desorption ionisation–time of flight mass spectrometry for rapid identification of Laribacter hongkongensis. J. Clin. Pathol. 2013, 66, 1081–1083.

Scilight Press

Author Information

Abstract

Keywords

References

About Scilight

Journals

Publishing Policies

Contact Us