Convolutional Neural Networks (CNN)

If an image is fed into an ordinary fully connected network, the number of parameters becomes excessive: a 224 × 224 color image has 150528 input values, and with 1000 neurons in the first layer about 150 million weights are needed. Such a network also ignores the spatial arrangement of pixels: neighboring pixels are treated the same as distant ones. Convolutional neural networks (CNN) solve this problem with local connections and weight sharing.

A convolutional layer slides a small filter (a kernel, for example 3 × 3) across the image, multiplies the pixels by the filter weights at each position, sums them and produces a feature map. The same filter is applied to the whole image, so there are few parameters and an object can be found wherever it appears in the image. Filters are not set by hand but learned during training: the first layers detect edges and corners, deeper ones detect shapes and parts of objects. Strictly speaking, libraries implement cross-correlation, but by tradition the operation is called convolution.

The main hyperparameters of a convolution are the filter size, the stride and the padding. If the input size is n, the filter f, the padding p and the stride s, the output size equals floor((n + 2*p - f) / s) + 1; for example, with n = 32, f = 5, p = 0 and s = 1 the output is 28. A pooling layer (most often 2 × 2 max pooling) shrinks the feature map, reduces computation and increases tolerance to small shifts. The result of a convolution is usually passed through a ReLU activation.

A typical CNN stacks blocks of convolution, ReLU and pooling, and at the end a fully connected layer and softmax produce class probabilities. Yann LeCun’s LeNet-5 (1998) recognized handwritten digits; AlexNet (2012) won the ImageNet competition by a wide margin and started the era of deep learning; ResNet (2015) made it possible to train networks of hundreds of layers with residual (skip) connections. When data is scarce, data augmentation and transfer learning, that is, adapting a pretrained model to a new task, are used.

Key takeaways

• Thanks to local connections and shared weights, a CNN uses far fewer parameters than a fully connected network.
• A convolutional layer with learnable filters builds a feature map; features become more complex with depth.
• The output size is found with the formula floor((n + 2*p - f) / s) + 1.
• Pooling reduces the size and increases tolerance to small shifts.
• When data is scarce, augmentation and transfer learning help.