Domain-specific embedding models for hydrology and environmental sciences: enhancing semantic retrieval and question answering
Water Science & Technology · 01.11.2025 · 92 (9) · 1328-1342
DOI: 10.2166/wst.2025.156 ↗Kurz gefasst
Fachartikel in der Zeitschrift Water Science & Technology, online erschienen am 25.10.2025. Verfasst von Ramteja Sajja und 2 weiteren Autor:innen. Beteiligte Einrichtung: IIHR - Hydroscience and Engineering, University of Iowa, Iowa City, IA 52242, USA. Hauptthemen laut MEDLINE: Hydrology, Environmental Science, Large Language Models. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 59 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Oceanic and Atmospheric Administration.
Abstract (Originalsprache)
Large Language Models (LLMs) have shown strong performance across natural language processing tasks, yet their general-purpose embeddings often fall short in domains with specialized terminology and complex syntax, such as hydrology and environmental science. This study introduces HydroEmbed, a suite of open-source sentence embedding models fine-tuned for four QA formats: multiple-choice (MCQ), true/false (TF), fill-in-the-blank (FITB), and open-ended questions. Models were trained on the HydroLLM Benchmark, a domain-aligned dataset combining textbook and scientific article content. Fine-tuning strategies included MultipleNegativesRankingLoss, CosineSimilarityLoss, and TripletLoss, selected to match each task's semantic structure. Evaluation was conducted on a held-out set of 400 textbook-derived QA pairs, using top-k similarity-based context retrieval and GPT-4o-mini for answer generation. Results show that the fine-tuned models match or exceed performance of strong proprietary and open-source baselines, particularly in FITB and open-ended tasks, where domain alignment significantly improves semantic precision. The MCQ/TF model also achieved competitive accuracy. These findings highlight the value of task- and domain-specific embedding models for building robust retrieval-augmented generation (RAG) pipelines and intelligent QA systems in scientific domains. This work represents a foundational step toward HydroLLM, a domain-specialized language model ecosystem for environmental sciences.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- IIHR - Hydroscience and Engineering, University of Iowa, Iowa City, IA 52242, USA.
- ByWater Institute, Tulane University, New Orleans, LA 70118, USA E-mail: ramteja-sajja@uiowa.edu.
- ByWater Institute, Tulane University, New Orleans, LA 70118, USA.
- River-Coastal Science and Engineering, Tulane University, New Orleans, LA 70118, USA.
ORCID-Profile: Ramteja Sajja, Yusuf Sermet, Ibrahim Demir
Förderung
- National Oceanic and Atmospheric Administration
Zitieren
APA
Sajja, R., Sermet, Y., & Demir, I. (2025). Domain-specific embedding models for hydrology and environmental sciences: enhancing semantic retrieval and question answering. Water Science & Technology, 92(9), 1328-1342. https://doi.org/10.2166/wst.2025.156
Vancouver
Sajja R, Sermet Y, Demir I. Domain-specific embedding models for hydrology and environmental sciences: enhancing semantic retrieval and question answering. Water Science & Technology. 2025;92(9):1328-1342. doi:10.2166/wst.2025.156
BibTeX
@article{Sajja2025,
author = {Sajja, Ramteja and Sermet, Yusuf and Demir, Ibrahim},
title = {Domain-specific embedding models for hydrology and environmental sciences: enhancing semantic retrieval and question answering},
journal = {Water Science & Technology},
year = {2025},
volume = {92},
number = {9},
pages = {1328--1342},
doi = {10.2166/wst.2025.156}
}
EUVIMED