1x1 Convolutions Demystified

In the early development of convolutional neural networks (CNNs), convolutions with kernel size \(3\times 3\), \(5\times 5\), \(7\times 7\) or even \(11\times 11\) are often used. In the more recent literature, however, \(1\times 1\) convolutions are becoming prevalent. In this post, I will try to explain what \(1\times 1\) convolutions are and discuss why they are used in CNNs.