EUVIMEDEUVIMED entdecken
Crossref

LC-ATSR: A Transformer-Based Approach for Semantic Retrieval in Lakehouse Data Platforms

Youfang Xu

Informatica · 23.04.2026 · 50 (11)

DOI: 10.31449/inf.v50i11.13673 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Informatica, online erschienen am 23.04.2026. Verfasst von Youfang Xu.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This study proposes a Lakehouse Collaborative Adaptive Transformer Semantic Representation Algorithm (LC-ATSR) to address the core challenge of semantic retrieval of unstructured data in a Lakehouse integrated data platform. The algorithm uses a transformer as its backbone and incorporates a dual-dimensional adaptive mechanism of storage attributes and data types to construct an integrated "preprocessing-representation-retrieval" framework. The core designs include a storage attribute-enhanced attention mechanism that integrates Lake storage tags into self-attention computation to adapt to heterogeneous storage characteristics; a lightweight semantic compression and multi-modal alignment module that reduces vector dimensions (from 768 to 256) while constructing a unified semantic space; and a Lakehouse incremental semantic indexing mechanism that dynamically updates the index based on LSH hashing. The experimental results show that the LC-ATSR algorithm achieves a P@10 score of 89.7% and an F1 score of 88.2%, significantly outperforming mainstream algorithms such as BERT, DPR, and RoBERTa. The single-retrieval latency was 18.3ms, the incremental index construction time was reduced by 68.4% compared with the full-data approach, and the accuracy fluctuation was only 3.2% in heterogeneous data scenarios. The retrieval system built based on this algorithm achieves a 99.8% functional pass rate, a throughput of 286 QPS with 500 concurrent connections, and a response time of 21.3ms, meeting the engineering requirements of enterprise data platforms and providing technical support for the value mining of unstructured data.

Zitieren

APA

Xu, Y. (2026). LC-ATSR: A Transformer-Based Approach for Semantic Retrieval in Lakehouse Data Platforms. Informatica, 50(11). https://doi.org/10.31449/inf.v50i11.13673

Vancouver

Xu Y. LC-ATSR: A Transformer-Based Approach for Semantic Retrieval in Lakehouse Data Platforms. Informatica. 2026;50(11). doi:10.31449/inf.v50i11.13673

BibTeX
@article{Xu2026,
  author = {Xu, Youfang},
  title = {LC-ATSR: A Transformer-Based Approach for Semantic Retrieval in Lakehouse Data Platforms},
  journal = {Informatica},
  year = {2026},
  volume = {50},
  number = {11},
  doi = {10.31449/inf.v50i11.13673}
}
Kostenlos starten →