What's changed: Deepened MLS-C01 Chapter 5 (output-loss pairing/Adam/training diagnosis, CNN/RNN/Transformer internals & task mapping, transfer-learning stages/Ground Truth/distributed training/mixed precision + tables, scenarios, FAQ, traps; ja figures)
5.1Neural Network Fundamentals
The foundation of deep learning—understand layers and neurons, activation functions (ReLU/sigmoid/softmax), backpropagation, learning rate/epochs/batch, and the vanishing gradient.
A neural network is built from neurons arranged in layers, adds non-linearity via activation functions, and learns weights via backpropagation.
5.1.1Structure and training
- Activation: ReLU (common in hidden layers), sigmoid (binary), softmax (multi-class probabilities).
- Backprop + gradient descent: compute loss, backpropagate gradients, and update weights.
- Learning rate/epochs/batch: too-high LR diverges, too-low is slow; epochs are passes over all data.
- Vanishing gradient: deep nets lose gradients; mitigate with ReLU and batch normalization.
Common on MLS-C01: multi-class output = softmax, binary = sigmoid, hidden activation = ReLU (resists vanishing gradient), and too-high learning rate = divergence/oscillating loss. If loss won’t decrease or oscillates, suspect the learning rate first.
Continue reading — free sign-up
You're reading the free preview. Sign up free to read this section in full, plus every chapter (including 4+) and all questions.