fig2

Figure 2. Schematic diagram of the benchmark evaluation process for domain-specific LLMs. Stage 1 (Dataset Preparation): public datasets (ChemBench4K, GAOKAO_Bench), the EvalScope-native GPQA-Diamond, and the custom GPU materials set (500 QA pairs) are assembled. Stage 2 (Format Conversion): the datasets are standardized into the EvalScope-compatible MCQ/QA format, while GPQA-Diamond is skipped as it is natively supported. Stage 3 (Model Preparation): local open-source LLMs (Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, ChemDFM-v1.5-8B, ChemLLM-7B-Chat) and remote API models (Qwen3-32B, DeepSeek-V3.2) are configured. Stage 4 (Evaluation Execution): evaluation is run through the EvalScope framework, recording accuracy for the four MCQ benchmarks and BLEU-1 for the open-ended gpumaterials benchmark. Stage 5 (Result Visualization): the logged metrics are parsed and presented as bar charts. GPU: Graphics processing unit; QA: question-answer; MCQ: multiple-choice question; GPQA: graduate-level Google-proof Q&A; LLMs: large language models; API: application programming interface; CLI: command-line interface; BLEU-1: bilingual evaluation understudy, 1-gram.