Abstract

we investigate joint super- vised and unsupervised learning in a large-scale setting by augmenting existing neural networks with decoding pathways for reconstruction.

1. Introduction

Unsupervised로 pre-training 한참 하다가 요즘엔 아무도 안한다. There are many (supervised-learning) technics which were not limited by the modeling assumptions of unsupervised methods’'.

Supervised랑 unsupervised를 엮으려는 시도들이 조금 있었으나 대량의 데이터에 관한 것이 아니었다. Zhao et al.^[1]이 Zeiler et al.^[2]의 “unpooling” operator를 사용해 “what-where” autoencoder(SWWAE)를 제안했고 CIFAR with extended unlabeled data에서 괜찮은 결과를 얻었으나, 그보다 더 큰 데이터에서는 그렇지 못했다.

Classification network의 일부를 떼어내고 그것을 그대로 뒤집(은 후 augmentation을 더해 얻)은 것을 decoding pathway로 쓰겠다.

↑ Zhao, J., Mathieu, M., Goroshin, R., and Lecun, Y. Stacked what-where auto-encoders. arXiv:1506.02351, 2015.
↑ Zeiler, M., Taylor, G., and Fergus, R. Adaptive deconvolu- tional networks for mid and high level feature learning. In ICCV, 2011.

[1] Zhao, J., Mathieu, M., Goroshin, R., and Lecun, Y. Stacked what-where auto-encoders. arXiv:1506.02351, 2015.

[2] Zeiler, M., Taylor, G., and Fergus, R. Adaptive deconvolu- tional networks for mid and high level feature learning. In ICCV, 2011.

[1]

[2]

둘러보기 메뉴

Augmenting supervised neural networks with unsupervised objectives for large-scale image classification

Abstract

1. Introduction