A comparative study of QLoRA and knowledge distillation for developing an AI for intellectual property consultation using Qwen3-4B
Journal of Intellectual Property Education and Research · 15.06.2026 · 14 (1) · 43-53
DOI: 10.12972/kosiper.2026.14.3 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Intellectual Property Education and Research, online erschienen am 15.06.2026. Verfasst von Yunje Song und 2 weiteren Autor:innen.
Abstract (Originalsprache)
This study developed a Korean-language intellectual property consultation AI based on Qwen3-4 B Instruct and compared the performance of QLoRA and Knowledge Distillation (KD) in a small-s cale domain data environment. The dataset used for fine-tuning consists of 200 consultation Q& As, evenly composed of 40 examples each from five source types—judgment documents, statut es, precedent cases, decision documents, and authoritative interpretations—drawn from the AIHub Intellectual Property Law LLM dataset. During the sample selection phase, cases involving i dentical or similar duplicates, question-answer mismatches, incomplete answers, or those not c onforming to the intellectual property consultation format were reviewed to finalize the dataset. T he data was randomly split in an 8:2 ratio using seed 42, resulting in a training set of 160 cases a nd a validation set of 40 cases. The final evaluation was conducted using five separate questions and reference answers not included in the training or validation sets. QLoRA was implemented b y training a LoRA adapter after 4-bit quantizing Qwen3-4B, whereas KD was performed by comp ressing knowledge using Qwen3-4B as the Teacher and Qwen3-0.6B as the Student. Under the se experimental conditions, QLoRA recorded a Cosine Similarity of 0.9720, a Perplexity of 15.177 6, and a Latency of 2.7610 seconds, while KD recorded a Cosine Similarity of 0.9358, a Perplexity of 49.0222, and a Latency of 32.0134 seconds. Therefore, QLoRA is interpreted as having a com parative advantage over KD in a small-scale legal consultation data environment. However, given the limited number of samples and final evaluation items, these results should be viewed as preli minary data identifying a promising candidate rather than definitive proof of suitability for practica l use.
Zitieren
APA
Song, Y., Kim, H., & Kwon, H. (2026). A comparative study of QLoRA and knowledge distillation for developing an AI for intellectual property consultation using Qwen3-4B. Journal of Intellectual Property Education and Research, 14(1), 43-53. https://doi.org/10.12972/kosiper.2026.14.3
Vancouver
Song Y, Kim H, Kwon H. A comparative study of QLoRA and knowledge distillation for developing an AI for intellectual property consultation using Qwen3-4B. Journal of Intellectual Property Education and Research. 2026;14(1):43-53. doi:10.12972/kosiper.2026.14.3
BibTeX
@article{Song2026,
author = {Song, Yunje and Kim, Hyundong and Kwon, Hyukhong},
title = {A comparative study of QLoRA and knowledge distillation for developing an AI for intellectual property consultation using Qwen3-4B},
journal = {Journal of Intellectual Property Education and Research},
year = {2026},
volume = {14},
number = {1},
pages = {43--53},
doi = {10.12972/kosiper.2026.14.3}
}
EUVIMED