This lesson is being piloted (Beta version)
If you teach this lesson, please tell the authors and provide feedback by opening an issue in the source repository

Introduction to deep-learning: Glossary

Key Points

  • Machine learning is the process where computers learn to recognise patterns of data.

  • Artificial neural networks are a machine learning technique based on a model inspired by groups of neurons in the brain.

  • Artificial neural networks can be trained on example data.

  • Deep Learning is a machine learning technique based on using many artificial neurons arranged in layers.

  • Deep Learning is well suited to classification and prediction problems such as image recognition.

  • To use Deep Learning effectively we need to go through a workflow of: defining the problem, identifying inputs and outputs, preparing data, choosing the type of network, choosing a loss function, training the model, tuning Hyperparameters, measuring performance before we can classify data.

  • Keras is a Deep Learning library that is easier to use than many of the alternatives such as TensorFlow and PyTorch.

Classification by a Neural Network using Keras
  • The deep learning workflow is a useful tool to structure your approach, it helps to make sure you do not forget any important steps.

  • Exploring the data is an important step to familiarize yourself with the problem and to help you determine the relavent inputs and outputs.

  • One-hot encoding is a preprocessing step to prepare labels for classification in Keras.

  • A fully connected layer is a layer which has connections to all neurons in the previous and subsequent layers.

  • keras.layers.Dense is an implementation of a fully connected layer, you can set the number of neurons in the layer and the activation function used.

  • To train a neural network with Keras we need to first define the network using layers and the Model class. Then we can train it using the function.

  • Plotting the loss curve can be used to identify and troubleshoot the training process.

  • The loss curve on the training set does not provide any information on how well a network performs in a real setting.

  • Creating a confusion matrix with results from a test set gives better insight into the network’s performance.

Monitor the training process
  • Separate training, validation, and test sets allows monitoring and evaluating your model.

  • Batchnormalization scales the data as part of the model.

Advanced layer types
  • Convolutional layers make efficient reuse of model parameters.

  • Pooling layers decrease the resolution of your input

  • Dropout is a way to prevent overfitting



External references

Here is a (non exhaustive) list of external resources for further study after this lesson:

Miscellaneous resources

Some ML challenges or benchmarks

Some courses for deeper learning: