Language model (LM) pretraining can learn various knowledge from text corpora, helping downstream tasks. However, existing methods such as BERT model a single document, and do not capture dependencies or knowledge that span across documents. In this work, we propose LinkBERT, an LM pretraining method that leverages links between documents, e.g., hyperlinks. 2022: Michihiro Yasunaga, J. Leskovec, Percy Liang Ranked #1 on Text Classification on BLURB https://arxiv.org/pdf/2203.15827v1.pdf
Version: 20240320
Comments (0)
To leave or reply to comments, please download free Podbean or
No Comments
To leave or reply to comments,
please download free Podbean App.