PhD Seminar • Systems and Networking • Distributed DNN Training on Serverless ResourcesExport this event to calendar

Friday, July 15, 2022 — 2:00 PM to 3:00 PM EDT

Please note: This PhD seminar will be given online.

Runsheng (Benson) Guo, PhD candidate
David R. Cheriton School of Computer Science

Supervisor: Professor Khuzaima Daudjee

Deep Neural Networks (DNNs) are often trained in parallel on a cluster of virtual machines (VMs) so as to reduce training time. However, this requires explicit cluster management, which is cumbersome and often results in costly over-provisioning of resources. Training DNNs on serverless compute is an attractive alternative that is receiving growing interest. In a serverless environment, cluster management is handled for the user, compute resources can be scaled at a fine-grained level, and users are billed only for resources that are consumed. Despite these advantages, existing serverless systems for DNN training are ineffective because they are limited to CPU-based training and bottlenecked by expensive distributed communication.

I will present Hydrozoa, a serverless system we have developed for distributed DNN training. Hydrozoa overcomes existing limitations of serverless DNN training with a novel architecture that combines serverless containers with hybrid-parallel training and supports dynamic worker scaling, which helps improve statistical training efficiency. Hydrozoa achieves significant throughput-per-dollar improvements over existing VM-based and serverless training approaches while relieving the user from the burden of managing machine clusters.


Bio: Runsheng (Benson) Guo is a PhD candidate whose research interests are in distributed ML training, serverless & cloud computing.


To join this PhD seminar on Zoom, please go to https://uwaterloo.zoom.us/j/92272133761.

Location 
Online PhD seminar
200 University Avenue West

Waterloo, ON N2L 3G1
Canada
Event tags 

S M T W T F S
31
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
1
2
3
  1. 2022 (168)
    1. August (27)
    2. July (23)
    3. June (17)
    4. May (20)
    5. April (24)
    6. March (22)
    7. February (16)
    8. January (19)
  2. 2021 (210)
    1. December (21)
    2. November (13)
    3. October (12)
    4. September (21)
    5. August (20)
    6. July (17)
    7. June (11)
    8. May (16)
    9. April (27)
    10. March (20)
    11. February (13)
    12. January (19)
  3. 2020 (217)
  4. 2019 (255)
  5. 2018 (217)
  6. 2017 (36)
  7. 2016 (21)
  8. 2015 (36)
  9. 2014 (33)
  10. 2013 (23)
  11. 2012 (4)
  12. 2011 (1)
  13. 2010 (1)
  14. 2009 (1)
  15. 2008 (1)