作者：Fangyu Liu Qianchu Liu Shruthi Bannur Fernando Pérez-García Naoto Usuyama Sheng Zhang Tristan Naumann Aditya Nori Hoifung Poon Javier Alvarez-Valle Ozan Oktay Stephanie L. Hyland
标签稀缺性是提高专用数据库任务性能的瓶颈。我们提出了一种新的用于零样本域传输的合成传输学习框架（DoT5域合成零样本T5）。在不访问域内标签的情况下，DoT5以多任务的方式联合学习域知识（来自未标记的域内自由文本的MLM）和任务知识（来自对更容易获得的通用域数据的任务训练）。为了提高任务训练的可传输性，我们设计了一种名为NLGU的策略：我们同时训练用于域内标签到数据生成的NLG，从而实现用于自我微调的数据增强和用于标签预测的NLU。我们评估了生物医学领域和放射学资源节约型子域的DoT5，重点是NLI、文本总结和嵌入学习。DoT5通过多任务学习证明了作文迁移学习的有效性。特别是，DoT5在零发射传输方面优于当前的SOTA超过7个绝对值
Label scarcity is a bottleneck for improving task performance in specialiseddomains. We propose a novel compositional transfer learning framework (DoT5 -domain compositional zero-shot T5) for zero-shot domain transfer. Withoutaccess to in-domain labels, DoT5 jointly learns domain knowledge (from MLM ofunlabelled in-domain free text) and task knowledge (from task training on morereadily available general-domain data) in a multi-task manner. To improve thetransferability of task training, we design a strategy named NLGU: wesimultaneously train NLG for in-domain label-to-data generation which enablesdata augmentation for self-finetuning and NLU for label prediction. We evaluateDoT5 on the biomedical domain and the resource-lean subdomain of radiology,focusing on NLI, text summarisation and embedding learning. DoT5 demonstratesthe effectiveness of compositional transfer learning through multi-tasklearning. In particular, DoT5 outperforms the current SOTA in zero-shottransfer by over 7 absolute points in accuracy on RadNLI. We validate DoT5 withablations and a case study demonstrating its ability to solve challenging NLIexamples requiring in-domain expertise.
论文链接：http://arxiv.org/pdf/2303.13386v1
更多计算机论文：http://cspaper.cn/