PhD Seminar • Data Systems — Dynamic Early Exiting for Accelerating BERT InferenceExport this event to calendar

Wednesday, February 26, 2020 12:15 PM EST

Ji Xin, PhD candidate
David R. Cheriton School of Computer Science

Pretrained language models such as BERT have brought significant improvement to NLP applications. These models are based on the Transformer architecture and are pretrained on large-scale unsupervised data. Despite their success, they are also notorious for being slow in inference, which makes it difficult to deploy them in real-time scenarios.

In the talk, I will introduce a simple but effective method, DeeBERT, to accelerate BERT inference by early exiting. It allows inference samples to exit earlier after going through only a part of the BERT model. Experiments show that DeeBERT is able to save up to 36% inference time while maintaining the same model quality. Further analyses show different behaviour of transformer layers in BERT, and also reveal BERT’s redundancy. 

I will also discuss different ways to train the early exit architecture effectively, and other problems encountered during the process. Our work provides a new idea to apply deep transformer-based pretrained models to downstream tasks.

Location 
DC - William G. Davis Computer Research Centre
1304
200 University Avenue West

Waterloo, ON N2L 3G1
Canada

S M T W T F S
28
29
30
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
1
  1. 2024 (97)
    1. April (20)
    2. March (27)
    3. February (25)
    4. January (25)
  2. 2023 (296)
    1. December (20)
    2. November (28)
    3. October (15)
    4. September (25)
    5. August (30)
    6. July (30)
    7. June (22)
    8. May (23)
    9. April (32)
    10. March (31)
    11. February (18)
    12. January (22)
  3. 2022 (245)
  4. 2021 (210)
  5. 2020 (217)
  6. 2019 (255)
  7. 2018 (217)
  8. 2017 (36)
  9. 2016 (21)
  10. 2015 (36)
  11. 2014 (33)
  12. 2013 (23)
  13. 2012 (4)
  14. 2011 (1)
  15. 2010 (1)
  16. 2009 (1)
  17. 2008 (1)