Classifying Clinical Evidence Levels of Cancer Variants in Biomedical Literature Using Machine Learning and Large Language Models.
Summary
This study compares the performance of large language models (LLMs such as GPT-4.1-mini and Gemini-2.5-Flash) and machine learning (ML) algorithms (decision tree and XGBoost) for classifying biomedical literature according to the CIViC evidence level system. LLMs were evaluated using zero- and few-shot prompting strategies, while ML models utilized TF-IDF and word embedding representations. XGBoost with TF-IDF achieved the highest performance (micro-F1 = 0.83), outperforming both LLMs and decision trees. All models performed best on mid-range evidence levels (B to D) but struggled with high (A) and inferential (E) levels. The findings suggest that abstract-level evidence classification is largely driven by explicit lexical cues, with limited additional benefit from standalone LLM-based approaches.
Analysis
This research is crucial for precision oncology as it aims to automate and accelerate the interpretation of genetic variants by classifying clinical evidence levels. Rapid and reliable evidence classification can facilitate more informed and personalized therapeutic decisions for cancer patients. Although LLMs have not yet outperformed traditional machine learning algorithms in this specific task, the continued exploration of these technologies is essential for developing more efficient and scalable clinical decision support tools, thereby reducing manual workload and improving diagnostic and treatment accuracy.