deep learning with python by françois chollet

Most of deep learning consists of chaining together simple layers that will implement a form of progressive data distillation.

Specifically, layers extract representations out of the data fed into them-hopefully, representations that are more meaningful for the problem at hand. Some data goes in, and it comes out in a more useful form. The core building block of neural networks is the layer, a data-processing module that you can think of as a filter for data.

Once again, it’s a simple mechanism that, once scaled, ends up looking like magic.

A network with a minimal loss is one for which the outputs are as close as they can be to the targets: a trained network. This is the training loop, which, repeated a sufficient number of times (typically tens of iterations over thousands of examples), yields weight values that minimize the loss function. This adjustment is the job of the optimizer, which implements what’s called the Backpropagation algorithm: the central algorithm in deep learning. The loss function takes the predictions of the network and the true target (what you wanted the network to output) and computes a distance score, capturing how well the network has done

To control something, first you need to be able to observe it.

Deep learning with python by françois chollet