In our lab meeting tomorrow, Hassan will introduce his work on Neural Machine Translation, which is also his submission of EACL 2021.
A Zoom link will be posted to Twist on the morning of the meeting.
Better Neural Machine Translation by Extracting Linguistic Information from BERT
Abstract: Adding linguistic information (syntax or semantics) to neural machine translation (NMT) have mostly focused on using point estimates from pre-trained models. Directly using the capacity of massive pre-trained contextual word embedding models such as BERT has been marginally useful in NMT because effective fine-tuning is difficult to obtain for NMT without making training brittle and unreliable. We augment NMT by extracting dense fine tuned vector-based linguistic information from BERT instead of using point estimates. Experimental results show that our method of incorporating linguistic information helps NMT to generalize better in a variety of training contexts and is no more difficult to train than conventional Transformer-based NMT.
Tuesday, October 27th, 09:30 a.m.
