EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Domain-specific embedding models for hydrology and environmental sciences: enhancing semantic retrieval and question answering

Ramteja Sajja iD, Yusuf Sermet iD, Ibrahim Demir iD

Water Science & Technology · 01.11.2025 · 92 (9) · 1328-1342

DOI: 10.2166/wst.2025.156 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Water Science & Technology, online erschienen am 25.10.2025. Verfasst von Ramteja Sajja und 2 weiteren Autor:innen. Beteiligte Einrichtung: IIHR - Hydroscience and Engineering, University of Iowa, Iowa City, IA 52242, USA. Hauptthemen laut MEDLINE: Hydrology, Environmental Science, Large Language Models. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 59 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Oceanic and Atmospheric Administration.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large Language Models (LLMs) have shown strong performance across natural language processing tasks, yet their general-purpose embeddings often fall short in domains with specialized terminology and complex syntax, such as hydrology and environmental science. This study introduces HydroEmbed, a suite of open-source sentence embedding models fine-tuned for four QA formats: multiple-choice (MCQ), true/false (TF), fill-in-the-blank (FITB), and open-ended questions. Models were trained on the HydroLLM Benchmark, a domain-aligned dataset combining textbook and scientific article content. Fine-tuning strategies included MultipleNegativesRankingLoss, CosineSimilarityLoss, and TripletLoss, selected to match each task's semantic structure. Evaluation was conducted on a held-out set of 400 textbook-derived QA pairs, using top-k similarity-based context retrieval and GPT-4o-mini for answer generation. Results show that the fine-tuned models match or exceed performance of strong proprietary and open-source baselines, particularly in FITB and open-ended tasks, where domain alignment significantly improves semantic precision. The MCQ/TF model also achieved competitive accuracy. These findings highlight the value of task- and domain-specific embedding models for building robust retrieval-augmented generation (RAG) pipelines and intelligent QA systems in scientific domains. This work represents a foundational step toward HydroLLM, a domain-specialized language model ecosystem for environmental sciences.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

HydrologyEnvironmental ScienceLarge Language ModelsSemantics

Stichwörter der Autor:innen

HydroLLMLarge Language Models (LLM)Retrieval-Augmented Generation (RAG)domain-specific embeddingsfine-tuningsemantic retrieval

Autor:innen und Institutionen

  • IIHR - Hydroscience and Engineering, University of Iowa, Iowa City, IA 52242, USA.
  • ByWater Institute, Tulane University, New Orleans, LA 70118, USA E-mail: ramteja-sajja@uiowa.edu.
  • ByWater Institute, Tulane University, New Orleans, LA 70118, USA.
  • River-Coastal Science and Engineering, Tulane University, New Orleans, LA 70118, USA.

ORCID-Profile: Ramteja Sajja, Yusuf Sermet, Ibrahim Demir

Förderung

  • National Oceanic and Atmospheric Administration

Zitieren

APA

Sajja, R., Sermet, Y., & Demir, I. (2025). Domain-specific embedding models for hydrology and environmental sciences: enhancing semantic retrieval and question answering. Water Science & Technology, 92(9), 1328-1342. https://doi.org/10.2166/wst.2025.156

Vancouver

Sajja R, Sermet Y, Demir I. Domain-specific embedding models for hydrology and environmental sciences: enhancing semantic retrieval and question answering. Water Science & Technology. 2025;92(9):1328-1342. doi:10.2166/wst.2025.156

BibTeX
@article{Sajja2025,
  author = {Sajja, Ramteja and Sermet, Yusuf and Demir, Ibrahim},
  title = {Domain-specific embedding models for hydrology and environmental sciences: enhancing semantic retrieval and question answering},
  journal = {Water Science & Technology},
  year = {2025},
  volume = {92},
  number = {9},
  pages = {1328--1342},
  doi = {10.2166/wst.2025.156}
}
Kostenlos starten →