Skip to main content

CZ4042 Neural Networks and Deep Learning

Course Summary​

This course teaches students about Neural Networks and how they can be used to solve problems. The course is taught by Prof Jagath for the first half and Prof Chen Change Loy for the second. Prof Loy works at MMLab@NTU and is a highly cited researcher, who is able to provide deep insights about the content that he teaches in the second half as parts of it pertain to his research areas. The topics taught include:

  1. Neurons and Activation Functions
  2. Regression and Classification
  3. Neuron Layers
  4. Deep Networks
  5. Model Selection
  6. Convolutional Neural Networks (CNNs)
    1. Pointwise Convolution
    2. Depthwise Convolution
    3. Batch Normalisation
    4. Transfer Learning
    5. Data Augmentation
  7. Recurrent Neural Networks (RNNs)
    1. Elman and Jordan Type
    2. LSTM
  8. Attention
    1. Visual Attention
    2. Seq-2-Seq
    3. Transformers
  9. Autoencoders
  10. Generative Adversarial Networks (GANs)

Workload​

The workload for this course is fairly managable but does get heavier towards the end of the semester due to the project. Prof Jagath provides all the tutorial answers with sample code for your reference so that you can go back and take a look at how he arrived at his answer. There are no tutorials for the second half but examples are provided as well.

Projects​

There are 2 projects in this course. The first is to design some simple neural networks to solve basic classification and regression tasks. This is pretty straightforward to do and should not take very long if done correctly. There might be some confusion regarding the questions as they change the dataset every year so the TAs are available to clarify any doubts.

The second project is more tedious as it is self driven and you have to pick a project from a given list or make up one of your own. You can make models in any field of deep learning as long as you demonstrate that you can come up with a project of sufficient depth and a model of sufficient complexity.

Things to take note of​

The final for this exam does require A LOT of manual matrix multiplication so do get familiar with using your calculator and keying in the values. You will not have enough time to finish the paper so do strategise how you approach it. Questions for the second half of the course are more qualitative so it is easier to score there. Do also strategise which other modules you are going to take with this one as being in a project heavy semester will take up way more time than you imagine as training models is a slow process.

danger

Even though the Final Exam for this course is open book, DO NOT PRINT ALL THE SLIDES. You will be wasting paper, your money and your time flipping through the notes to find the piece of information you are looking for. Be smart and write down essential information on a cheatsheet no more than 6 pages (3 sheets of paper) and remember where the information is written. The course is called Deep Learning for a reason 🤦‍♂️.

Conclusion​

This course is a must take for all DSAI students as it teaches you all the important aspects of neural networks as well as introduces you to multiple network models that are used to solve specific problems.