PhD Seminar • Artificial Intelligence | Computer Vision • Mutual Information Maximization for Discriminative (Deep) ClusteringExport this event to calendar

Tuesday, April 5, 2022 — 1:00 PM EDT

Please note: This PhD seminar will be given in person.

Zhongwen (Rex) Zhang, PhD candidate
David R. Cheriton School of Computer Science

Supervisor: Professor Yuri Boykov

Maximizing information between network input and output is a basic principle for unsupervised training of discriminative posterior models, where it might be as fundamental as the likelihood maximization principle for the generative models, e.g., GMM. However, only a handful of unsupervised information-based techniques exist in computer vision, in part due to limited understanding of the corresponding losses.

We provide a systematic summary of existing related loss functions and motivate our self-supervised loss where pseudo-labels naturally appear as auxiliary variables when splitting mutual information. We also derive an efficient EM algorithm with closed-form solutions for E and M steps. Empirically, our approach is more robust compared to earlier methods for information maximization. Also, we conceptually illustrate that the classifier norm regularization for margin maximization must be explicitly present in information maximizing losses, which otherwise give degenerate clustering.

Location 
MC - Mathematics & Computer Building
2038
200 University Avenue West

Waterloo, ON N2L 3G1
Canada
Event tags 

S M T W T F S
29
30
31
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
1
2
  1. 2022 (123)
    1. July (5)
    2. June (17)
    3. May (20)
    4. April (24)
    5. March (22)
    6. February (16)
    7. January (19)
  2. 2021 (210)
    1. December (21)
    2. November (13)
    3. October (12)
    4. September (21)
    5. August (20)
    6. July (17)
    7. June (11)
    8. May (16)
    9. April (27)
    10. March (20)
    11. February (13)
    12. January (19)
  3. 2020 (217)
  4. 2019 (255)
  5. 2018 (217)
  6. 2017 (36)
  7. 2016 (21)
  8. 2015 (36)
  9. 2014 (33)
  10. 2013 (23)
  11. 2012 (4)
  12. 2011 (1)
  13. 2010 (1)
  14. 2009 (1)
  15. 2008 (1)