[人工智能]DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations

Abstract

• DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations
  • 用来学习通用 sentence embeddings 的自监督目标，不依赖于标注数据.
• 灵感来自于近期 deep metric learning (DML) 领域的进展
• 缩小了无监督方法和有监督方法之间的 gap
• 效果随着参数的增加和无标签训练数据的增加而提升
• 代码及模型：https://github.com/JohnGiorgi/DeCLUTR

1 Introduction

sentence embedding:

• 有监督方法：(Conneau et al., 2017)
• 半监督方法：(Subramanian et al., 2018; Phang et al., 2018; Cer et al.,

了解本专栏 订阅专栏 解锁全文