… applying the model to document retrieval. In this … pretrained transformers that share similarities
with BERT. We find that these BERT variants are not more effective for document retrieval …

… To show the generality of our approach we present tests conducted also for other pretrained
transformers in Section 6.5: a version of BERT that was more effectively pre-trained, ie, …

… Historically, document expansion techniques have not been as popular as query expansion
… interest in document expansion in the context of transformers, which we cover in Chapter 4. …

… In recent years, large pre-trained transformers have led to substantial gains in performance
over traditional retrieval … Does CE better rank the same documents retrieved by BM25? …

… [16] use pretrained contextual embeddings, without fine-tuning, in downstream ranking … In
this work we proposed a solution to apply Transformers to full document re-ranking. Our TKL …

… from Transformers. We design an unsupervised method to pre-train HIBERT for document
modeling. We apply the pre-trained HIBERT to the task of document summarization and …

… , we pre-train the Transformer model towards the supervised … At the finetuning stage, we use
our pre-trained Text Encoder … for effective web-document retrieval. Applied Intelligence 18, 3 …

… We consider the large-scale query-document retrieval problem: given a query (eg, a …
Transformer model, we compare the pretraining tasks to two baselines: No Pretraining and MLM. No …

… answer to a given question in relevant documents, eg, retrieved by a search engine. The
latter finds an exact text span in a document (or typically a paragraph) containing the answer. …

… Specifically, unlike the in-batch negative documents used by the MedCPT retriever that …
documents are sampled from rank e to rank f in the top retrieved documents by the pre-trained …